Unprecedented load.
replyFrom the GitHub COO on April 3rd:
Platform activity is surging. There were 1 billion commits in 2025.
Now, it's 275 million per week, on pace for 14 billion this year if
growth remains linear (spoiler: it won't.)
GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week
in 2025, and now 2.1B minutes so far this week.
So we're pushing incredibly hard on more CPUs, scaling services, and
strengthening GitHub’s core features.I find this hard to believe in this context. They should be utilizing load shedding or admission control and killing/rejecting jobs rather than hard failures if it is a scaling issue. It's much more likely an actual software defect than just more load. If this was the case (which they would likely prefer) free/public tiers would be removed first to preserve paying customers services.
replyWhy not both? Higher base load combined with insufficient internal controls for ratelimiting/load-shedding (as in, they don’t know who to shed) would be explanatory.
replyIf they can't implement something as simple as "decode upstream headers and determine if 429/503" I don't know what to say. Since this has knocked out all customers it indicates they likely don't have anything of this form implemented.
replyI meant shedding of legitimate base load, not retries. I think we can safely assume they do the latter.
reply[dead]
replyThey’ve said that, but it doesn’t add up. The outages started shortly after the migration to Azure became the top priority, and before the load massively increased form agentic coding (by their own dates).
replyI guess the AI Slop is that bad in terms of increasing commits/actions/etc
replydeleted
reply