It's a simulation; why should clock time be involved to begin with? The only things that should care about clock time are those that exist outside the sim, e.g audio
But I’m not 100% that’s even true; in the context of replay, I imagine it’d be more appropriately part of the sim for it to scrub properly.
In the context of networked games with client-side prediction, I think it’d probably be key frames tied to logical time and intermediate frames tied to wallclock