Is it a huge deal if I already put interior poly on exterior trim?

You don't need to strip it. Interior poly and exterior poly are both urethane finishes, so the new stuff will bond to the old stuff fine as long as you scuff it up first. I've recoated over interior varnish more than once on covered porches and it held up okay, but there are a couple things to know before you decide to just leave it.

The real weakness of interior poly is sun, not water. It's got no UV blockers in it, so any casing that catches direct light will amber, crack, and flake way faster than the shaded parts. Under a covered porch with mostly indirect light you'll probly get a few decent years out of it. But you said rain blows in from the side, and that's the other weak spot. Interior poly handles the odd wetting fine, it's the repeated wet-dry cycles pushing moisture into the end grain and behind the finish that'll lift it. Pay extra attention to the bottom of the casings and any end grain, that's where it always fails first.

What I'd do is give everthing a light scuff sand with 220, just enough to knock the sheen off so the new coat grabs, then wipe the dust off with a tack cloth or a rag dampened with mineral spirits. Then put two coats of an exterior spar varnish or exterior poly over the top. Spar varnish stays a little flexible so it moves with the wood when the door casing swells and shrinks, which matters on a back door that sees weather. Follow the recoat window on the can so the coats bond to each other.

If you were going to see full sun and direct rain on this trim I'd say strip it and start clean, no question. Under a covered porch with a light sand and two proper coats over top, you're fine. Worst case you're recoating in four or five years instead of ten, which you'd probably do anyway.

Agree with the recoat plan here. I did almost the same thing on a shed window trim a few years back, two coats of the wrong can before I caught it. Sanded it down, put exterior finish over top, and that side of the shed faces weather and it's still holding six seasons later. One thing I'd add, don't skip the back edges and any cut ends of the casing, slap some of the exterior stuff on those too since they're hidden til the casing is on. Mine started going soft at an end grain spot first and that was the part I rushed.

Glad I read this thread because I did the same thing on a bench I built for our back step last summer. It's under a roof but gets some rain on it when the wind kicks up, and so far the finish still looks okay, but now I'm wondering if trouble is coming. What does it actualy look like when the wrong stuff starts failing? Is it peeling paint style or more of a dull cloudy look? I'd like to know what to watch for before it gets ugly.

Kind of relieved to see this thread, I did basially the same thing on a garden gate last spring except I never caught it at all. Two years on and it still looks decent, though the gate gets way more sun than your trim so I'm half expecting it to let go eventually. Question for anyone who's done the recoat trick though, my understanding was you have to let the first finish fully cure before topping it, and mine went on the shelf alredy half hardened. Does that change anything, or is it the same light scuff and go ahead situation regardless? Don't want to make a second mistake on top of the first one.

Update: took the scuff-sand advice and put two coats of the exterior stuff over what was already on there. Seems to have grabbed fine, no wierd spots so far. One thing still bugging me though, one of you was about to explain what it looks like when the wrong finish starts letting go and the reply cut off. Is it peeling at the edges first, or does it go cloudy?