What is the source of the roughness? Paint fragments or small wood fibers that popped up from the surface?
You may well have "raised the grain"- caused such small fibers that remain after sanding to stand proud after getting wet.
Were that so, I'd be sure to use a cabinet scraper (rather than any sander) prior to sealing and between subsequent coats.
Can easily pass the "baby's bottom" test, doing that, with construction-grade plywood- utility stuff.
TTFN, J