Only new queens are left from the old colony and they aren't in the nest.
What you call vermin are creatures which serve a useful purpose (in our human terms) by getting rid of what's not neded any longer.
If there's no old nest there mice will find somewhere else to go, you can't blame the presence of mice on an old nest.
Mary