It's technically possible to replace rotating mass with batteries using a "grid-forming inverter", which is an inverter that converts the battery DC to AC with frequency varying depending on the grid load, simulating how that rotating mass would behave ("synthetic inertia"):

https://en.wikipedia.org/wiki/Inverter-based_resource#Grid-f...

This competes with the traditional giant flywheel option ( https://en.wikipedia.org/wiki/Synchronous_condenser ), which has the advantage of being a simple and proven technology, and handling brief overload better, but the disadvantage of having moving parts. It's not clear which option is currently best. Both are in current use.

And as a third answer: You actually don't need to take the flywheels of the power plants offline, when you take the rest of the power plant down. The locations for them are perfect, as the whole grid has historically been build around them.

We actually don’t need those anymore. Grid forming inverters and batteries will take over that role.