There's some merit in that but if you lose a non-contactless card, the finder can't use it without knowing the PIN (or your address for online transactions). If you lose a contactless card, the finder can make quite a few purchases of up to £30[1] without needing to enter a PIN.
Whilst the card issuers claim that they will reimburse fraudulent use, how are you going to prove that it wasn't you who made the transactions?
[1] I think that's now the limit. It started a lot lower and has increased several times. Will probably be £100 before long!