I've just lined a bay with celotex and all seems fine - except the window board/sill.
At the moment it's 25mm wood of some sort, and sits on top of a brick wall - possibly cavity, difficult to tell. It measures just a couple of degrees above the outside temperature. The newly insulated area is pretty much room temperature.
The maximum thickness - informed by the UPVC window frames - is 35mm. Is it worth looking at insulating this area - say, 25mm of PIR faced with 9mm ply? The finish wouldn't be as good - and wouldn't be as robust as the existing board. There doesn't seem to be any insulated board for this purpose.
Or just let it go, and accept this is the way it is.