Yes, they kill/eat most of the harmful garden insects. Wasps are also pollinators of many plants.
I would never normally get rid of a wasps nest. I've done it only a couple of times - once where a nest was a nuisance to a neighbour who is allergic to stings, and another time when they built one over my parents back door. We watched it with much interest for months as it grew, but it started dropping lots of half-dead wasps out which the cat had to step across to get in/out of the cat flap, and at that point it had to go.