As I swapped the bulbs one by one all seemed normal until I unplugged the last halogen at which point the other four LEDs started glowing dimly. Replacing the halogen bulb turned the LEDs off again.
Now with all five bulbs in they all continue to glow dimly when "off" and work normally when on.
How do I fix this? It's a one-way switch that only controls one light. It is next to a two-way one that controls the upstairs hall light.
(I've just replaced all the halogens with LEDs in the upstairs hall light but it doesn't display this phenomenon.)