The problems for services such as GitHub with scaling are reducing cost per customer. That's even more pertinent when discussing inference at scale.
reply> The problems for services such as GitHub with scaling are reducing cost per customer. That's even more pertinent when discussing inference at scale.
replyI don't think that's true. When I look at GitHub's incident history,[0] it doesn't read to me like a company that's struggling to cut costs. It looks like a company that's trying to do a million things to serve a million use cases, and the growing interconnections between all those distinct services and workflows cause unexpected failures.