Hey Morten,
It looks like you are trying to mirror the surface result and look for smoothness between the 2? These ends are not planar here, which will create 2 surfaces that deviate from tangent. If you look in the top view, fix these to be planar also and the result surfaces should be smooth.

|