What if <video> and <audio> had a new attribute scope with possible values page and session?
scope="page" would be the assumed default and cause the media player to work as it does today.
scope="session" would preserve the underlying runtime state of the media player across page loads in the same session allowing uninterrupted audio playback and scriptability.
There’d be a lot of details to work out, but that’s the basic idea.