can I apply exterior semi gloss directly over flat exterior paint?

Go right over it , no primer necessary.

Sure, but first you need to get all that "dust, pollen, etc." off or you will just be painting dirt.

There's a saying that nothing sticks to latex like more latex.

But having said that, you can run into problems if the base coat is a real low end/non acrylic latex.

You'll probably want 2 coats anyway so I'd make the firt one a primer coat tinted to half the color formula of the topcoat and then you can probably get by with one topcoat assuming you use quality paint.

I prefer Pratt and Lambert Suprime for the primer and Accolade for the topcoat. Make sure you start by preparing the walls by washing with a TSP solution and rinse.

Paint is fine, dirt is not. Powerwash it first, and remove any peeling paint obviously. Otherwise, flat paint essentially is primer (although it's not stain blocking of course.)

I have never understood this advice. Just use the exact same color.

Tint the primer so if coverage is not totally opaque there is less contrast with the top coat. Primer can't be tinted as much as cover coats or it reduces the primer adhesion/sealing qualities, according to my favorite paint store salesman.

Well I suppose if you're painting your walls red, it might be a problem. Otherwise, just use the exact same good quality paint you're going to use anyway (unless you need stain blocking or something, of course.)