Against stinging insects in general, I keep a household spray bottle handy with a mix of 1:15 soap to water. I use dish soap like Dawn or something similar. Water alone will not kill them, they are adapted to outdoor living and rain. However soapy water drowns them quickly. Hit with this spray, they die FAR faster than with Raid, and I don't have to worry about poisoning the kids or the dogs.
But I haven't wanted to waste expensive dish soap on nests in the ground. I just throw in a cup of tide or similar laundry detergent to each bucket of water.
Soapy water wets them and drowns them. Plain water doesn't, I guess they must have some way of shedding it. I don't really know the physics of this, but I have done it a number of times and can testify it works.