When you buy pop rivets you typically have to specify the material for
both the rivet and the mandrel. The rivet is what goes into the hole,
and the mandrel is what squashes the rivet in the hole and breaks off
once the rivet is fully squashed.
If you specify both an aluminum rivet and an aluminum mandrel, there
won't be anything in that rivet to rust. If you then use a bit of
caulking over the rivet and the joint in the gutter, as Trader
suggested, you've got a leak free connection.
Marson makes both good quality pop rivets and a top quality pop rivet
tool for this kind of work. If you order a Marson pop rivet kit online
you should be able to get the kit at a good price. And, most places
listed under "Fasteners" in your yellow pages directory will sell rivets
either by the box or individually.