That is more MOO (Mutuality of Obligation) than working hours specifically.
Well kind of, in the sense there is no obligation to accept work at a certain time, but there is also the freedom from the entity offering it to not offer if it can't be done at a time that suits them as well.
As you say, often the nature of the work will dictate times. e.g. not much use for a freelance camera man that can't be there at the same time as the rest of the cast and crew. Which makes the existing case law hard to apply in many circumstances.