Yes. When I visited as a teenager the very first UK-FR undersea link, the engineers said 'if we switch it off and disconnect it, we can draw an arc for half an hour from the stored electricity'.
Seawater is a good conductor, unlike air, so the cables must be insulated, and that insulation forms the bulk of the dielectric.
That's why they had to be DC - charging and discharging that capacitance via a resistive cable cant be done 50 times a second without massive losses.