I know this sounds awful and tedious, but I think the only way to solve this
as someone said "remove it all".
I guess I would use a box cutter blade and razor it out on the side of
even some plaster edge. Remember, don't let remaining silicone destroy your
best effort to fix this now, so be thorough.
And if you are comfortable with mud, just replaster the corner seam.
It may require 3 coats, so be it. Let it dry completely.
Then sand it using a plaster sanding block. They work great and make a nice
if you stick with a 120 grit. Just be warned, expect lots of fine dust.
It's unfortunate, but you painted yourself into a corner, so to speak.