Yes your piece part on the right is not very clean!

Better to redraw it!
Hide the Cylinder
Kill the internal circular face, kill the circles edges, redraw a rectangle face, Join for a solid
Unhide
and make your boolean Diff