There have been other 4D video games with actually good performance though.

They should have used Gaussian Splats, surely this would reduce processing overhead.