upvote
> If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended?

Assuming the timestamps are accurate, Google probably started terminating resources while the account was not "suspended" and only completed that after all resources were disabled.

reply
Or the account started doing something nefarious (assuming one of their customers as root cause, not railway itself) that started causing real problems and Google shut it down.

The problem with not having the data is that it’s easy to make assumptions.

reply
The absence of any explanation for the suspension does seem intentional. If it were me that's one of the first things I would've asked so that I could make sure it doesn't happen again.
reply
The 22:20 timestamp from the body of the post is wrong. The timeline section (where the 22:10 timestamp came from) is consistent with itself, and also contains:

> May 19, 22:19 UTC - Root cause identified: Google Cloud Platform has suspended Railway's production account.

They couldn't have identified the root cause before it happened.

reply
That 10 minutes is likely very normal. Possibly...

* A Google employee messes up a setting (like one of the previous incidents) triggers something that looks like a suspension is warranted and it takes 10 minutes to flow through the process to suspend.

* A Railway customer does something corrupt, or seemingly corrupt, Google's system starts limiting access and take 10 minutes to decide it should be a suspension.

These are even more likely if there is a person in the loop to approve, who obvious did not dig deep enough to see that they should not have done so.

reply