So...I replaced the thermocouple. Every so many years, it seems to need it. However, this did not solve the problem.
Preheating the thermocouple made the pilot stay on. For about two minutes.
I can't really see into the pilot compartment when I'm holding the knob down. Using a mirror did not help much. It looks as if the pilot flame is not hitting the thermocouple; it's "leaking" to the right side.
The pilot unit is pretty standard stuff, the thermocouple is on the left, parallel to and separated from the pilot by about half an inch. The whole assembly is rotated about 20 degrees from vertical, about 11 o'clock. The pilot orifice faces what I would call a "deflector" that bends the flame towards the thermocouple and also has a "v" assembly that routes the flame around the thermocouple.
It looks as if the flame is not being directed towards the thermocouple and is just randomly burning around the deflector. I have cleaned this as best as I can, poked and prodded, chipped and scraped, with no help.
The only thing left I can think of is that the orifice itself is partially obstructed. Is there any way to clean this without damage and without disassembling the whole heater? Is there something else that would cause this problem?