I’m not saying not to try or that there’s no value in turning the wheel once more. But I used to be excited about these kinds of things and now I’m not because I’ve seen the cycle a few times now.
But there are so many hooks into the Git workflow today, and I’m struggling to find a “user story” that would not be met by custom Git tooling. UX could be added to Git objects à la Gerrit with custom CLIs, custom Gitoxide ODB and (eventually) ref storage backends can help the server scale, and things like promisor remotes and partial clones can keep things smaller on the client side.
I’m curious what ERSC is able to share publicly about where the scaling fails with the Git specification? Is it the wire format, or the relative speed of upstreaming new things to Git? Or is the side-channel approach for storing rich data in Git objects insufficient in some way?
P.S. I wrote a blog post[1] about this a bit ago. I’m sure I’m missing something. Since I wrote it, I’ve wondered if the lack of deferred conflicts in-tree would be a scaling limiter.
Yes, it's important to realize that git is multiple things: the on-disk repo format, the CLI, and the wire protocol. Each has their own challenges, just like any project, but they can be tackled independently. But at the end of the day, there just are fundamental limits to the way that git does things that make total sense in the context Linus developed git for, but just do not in other ones. Some of those are performance related, but some of those are things like how the kernel is an open source project, and so per-directory ACLs are something that git wouldn't ever bother supporting, but are valuable for companies. We'll be talking about these things in the future, so I'm just going to be hand-wavy about it right now. But we do think that git is a really great system that has served us all well for 20 years, but just like how git displaced systems before it due to significantly re-thinking the details, for some users, making meaningful improvements over git will also require a similar significant re-think. That's just the way of technological development.
But that's good to hear.
Normal Git/JJ is fine for most people, and most open source projects. It's just when you get to the scale of commercial software companies (100+ employees) you run into the classic monorepo/submodule problem.
Monorepos are currently the best solution - they make testing tractable, you can do cross-project changes, Git's support for them isn't buggy as hell. But they have obvious downsides - performance is the big one, but also integration with open source projects, permission control, etc.
So there's this big problem space that really only matters to companies where as far as I know there are currently no good open source or even commercial solutions. The only people that have (maybe) solved it are the mega-corps like Facebook and Google who have made their own private solutions. I don't know exactly what ERSC are going to offer but it's surely going to be in that space and also probably not going to be super relevant to private individuals.
Our moat is that we are not trying to take git and scale it up, we are building a more scalable system from the beginning. And we will have features that nobody else has because of it.
It's always the case that businesses have competitors, that's just capitalism. We're not worried about that.
You mention performance issues with Git, but there are performance issues with this blog post.