Even after years of booleaning, I can still get stumped. Why isn't this fellow feeling very happy?

I'm going to do some surface trimming and joining to get them together, but it's going to become part of a much larger assembly, and I'm concerned there's some underlying weirdness that might cause trouble when I hours into further booleaning.