In the meantime I tried this workaround that seems to be okay. Instead of extruding a circle and then boolean merge my shape to it I just extruded the curve, applied regular fillet and then chamfered. Next step was a flow command and then another boolean merge to fix the minor distortion. The bend gives the fillet a little wonky-ness but at least I can move forward.