What do you do differently that you expect to have better performance than an experienced, established player?
replyNot GP, but just being smaller makes it easier to achieve reliability. Like if you're a git forge with 100 similar customers, you can likely achieve an order of magnitude better reliability than GitHub, who is trying to serve millions of customers with wildly different needs.
replyThe problems for services such as GitHub with scaling are reducing cost per customer. That's even more pertinent when discussing inference at scale.
reply> The problems for services such as GitHub with scaling are reducing cost per customer. That's even more pertinent when discussing inference at scale.
replyI don't think that's true. When I look at GitHub's incident history,[0] it doesn't read to me like a company that's struggling to cut costs. It looks like a company that's trying to do a million things to serve a million use cases, and the growing interconnections between all those distinct services and workflows cause unexpected failures.