How can I tell whether my water heater has a check valve?

It looks like a brass T with a hex end nut ontop, see the picture attached here. They actually do save a little bit of money by preventing the hot water from going too far from the heater so it does not have to reheat the water going into the pipe, colling off and coming back in when you are not using hot water. But, that's exactly what makes you wait that long for the hot water - all the water that's standing in the pipe is by definition cold if you are using the check valve.

I would only remove it if you are using hot water very frequently. If this is only a once-a-day inconvenience when you're doing your dishes, I would just let it drain for however long necessary instead of heating all that water all day long for no reason. Perhaps not a big deal in winter but definitely a waste in the summer.