> But seems something is systematically going wrong at GH
replyYes, we call it: Microslop.
It's honestly insane how terrible their reliability is. Over the past 4 weeks, 8 days with GitHub Actions outages, many of them multi-hour outages: >3 hours on each of July 9th, July 20th and today, and 1.5 hrs on July 23rd.
replyOutages happen, but this many outages so close together, and so many of them so major/long lasting, something is systematically wrong for sure. It's been seriously hamstringing our ability to ship code at my company.
Mon and Dad (MS) are (generally) pretty solid with uptime.
replyWhat is happening at GH?
Rate of change trying to keep up with new challengers? Over-reliance on AI? Engineers trying to debug slop?
Yeah who knows, would be interesting to hear an inside take if any readers here are also GH devs!
replyThey're at 93.91% uptime over the past 90 days, according to https://mrshu.github.io/github-statuses/ , and that doesn't even include today's outage yet.
A glorious one nine of reliability.
Yea just try requesting an SLA credit from them. According to THEIR numbers it’s 99.995%. It’s intolerable and I am going to put my full weight behind stripping as much work as we can from GHA as possible, even if leaving GitHub itself is effectively logistically and contractually impossible.
replyI see two nines in there…
replyDon't throw shade, there are two nines in that percentage for now. Miles a part though.
reply911 has one! :)
reply> What is happening at GH?
replyIn large part, the move from AWS to Azure. Azure's just bad.
Based on previous posts I've seen about this, IIRC the timing seems to imply that it has more to do with them being hammered with AI slop than it does with the Azure transition. Who really knows though
replywasn't that years ago? or was that Skype? hahaha
reply"most of us have been there done that" - so true. That feeling in your gut when you realize something you just did caused an outage is pretty unique.
replyThey're one rewrite in Rust away from fixing everything. (jk)
replyTime to touch some grass. Better use of my time and energy than twisting the remaining things on my todo list today to make more progress on them than I have managed. I should have taken a long lunch but I rebased the hell out of a PR instead.
reply