Bonjour Buzhug, Pilou,
Yes, the problem is in the area that Pilou points out - there are some very skinny surfaces in this area, where a very small shelf on the side approaches the main part within 0.0002 units . This type of situation tends to confuse the boolean operations, at around this distance it is better if the shelf is either exactly flush with the object, or a larger distance away.
This area of the model is a little confusing to me in general, I can't quite tell if the little shelf sticking out is important to your model of it is more of an accidental thing left over from some other boolean operations.
In general it is not necessary to start over again, there are techniques that you can use to edit individual pieces of your model to fix them up.
To do this, you would select some of the faces that are messy, and then do edit/separate - this will detach those faces from the main object, at which point you can hide the main object to get it out of the way. Then you can select this sub-structure and do edit/separate on it again which will now break it up into individual surfaces.
But I'm not quite sure if that little shelf is important to you - if it is not, then you would want to delete those little slivery surfaces and reconstruct the side with more of a simple single planar piece or something.
Another tool for fixing up a shape is untrimming and retrimming - this can allow you to fix up a trim boundary that isn't quite right. Often times for this you might select the existing edges of the surface and duplicate them with copy/paste, then hide them. Then select all the edges of the surface and hit delete - this will remove the trim curves from the surface and recover just the underlying surface. Now you show the original trim curves, make any edits to them to fix them up, and then re-trim the surface with the edited curves.
After doing these tune-ups or reconstructions to individual pieces to fix things up and remove little messy areas, you can then use Edit/join to merge them back together into a solid.
I would do some of these steps on your model, but like I mentioned, I'm not quite sure what your intent is with that little shelf area that is sticking out.
- Michael
