To remove the remainder of the sprue, you just take an ordinary file and file it off. The nearer you get, the lighter you go. Change the filing angle ever so often to avoid file marks. Once you are almost there, change to sanding paper wrapped around a stip. 400/800/1200 and it should be as polised as the rest.

For a next production run, ask your caster *NOT* to tumble it. A second look at the picture makes me believe to see an orange peel surface, sure sign that he tumbled in a rotary tumbler, and for too long. Build yourself a magnetic polisher with needles, and the surface should be much better.