Google apparently has custom chips that allow them to have the 1m context window.