I exclusively use gemini models and this has been my experience.

I mitigate it by creating dense planning docs for everything and executing iteratively.

Lot's of time wasted on procedure unfortunately

We know from NVIDIA's public Vera Rubin inference engine marketing materials that the frontier lab models are ~1-2T total.

Mythos is an exception that's larger.