When I was responsible for backups we kept the tape cartridges in a fire safe in a different building. We took a full backup weekly and moved the tape from the robot to the firesafe as soon as the backup was complete. Only the daily incremental backups stayed in the robot for more than a day.

> spinning up a new shard takes a quarter

Both can be true

You would need to understand first how they gained access and verify that they can’t do the same again. That in itself could take days if not weeks. Then of course they might have found new vulnerabilities while they were in, so you would need to worry about that too.

Yes, even hundreds of them sometimes.

The most time consuming part of recovering from an attack is validating everything. It takes more than a few hours to validate the infra that stays put isn’t compromised, the IaC code itself isn’t compromised, deploy the infra, bring a copy of the offline backup of your data (your IaC can’t drive to another site and bring the backups, then make a copy, unless tou are really sure you removed any trace of compromise), validate that the backup is sound, then restore it.

In some cases the infrastructure part is the least time consuming. Some platforms are straight forward enough that even manual deployment is fast. But after a hack you can’t trust anything so you need to do the slow validation that takes longer than your projected “hours”.