This solution with WAL and compact resembled the database internals as taught by Martin Kleppmann.
I asked Claude about this observation, and learned that the author (Vicent Martí) wasn't only core Github developer but also worked on Vitess from PlanetScale.
So no surprise these two contexts resulted in the design of Continuity as it is.
https://vitess.io/ - Vitess is a database clustering system for horizontal scaling of MySQL.
<edit>: there is lots of demand for a GitHub alternative due to poor uptime, poorly implemented components (CI), and Microsoft stewardship. Switching, especially at this early stage requires a huge component of trust.
He still pushes his agenda, he still has lots of followers who believe his nonsense
I feel like the article is a bit light on the design of the WAL and maintenance operations, but maybe it's me.
Anyway, I'd love to have a friendly chat with the author.
Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]
1: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu...
How many startups and corporations built on top of S3 semantics?
Snowflake and the big lakehouses are one of them. They use S3 as giant disk in the sky.
The first step is to separate the problems. The content addressable object storage and storing the refs. libgit2 has two backends for those already.
Then use appropriate distributed dbs for the two backends. With large scale something like Cassandra for the objects (AP) and FoundationDB (CP) for the refs (or the equivalent Amazon services).
For performance the objects need to be cached locally and requests routed to nodes where the objects are cached locally.
Doesn't 3PC require all nodes to agree, not just a majority?
IE - the interactive diagram and note about latency being bounded by the slowest replica
Not sure the last month GH had fewer than five minutes of downtime - but I think it's been a while
They should have just failed to scale git and leave place for the less idiosyncratic bazaar or mercurial to take the lead. These were slower, but would probably have scaled much better in the long term.
Github's core innovation was re-centralizing a decentralized SCM (as per article) by giving it nice web UI and features such as PRs. Oh and also free code hosting space. None of which relied on git's peculiarities.
I've been working on another project where I have binary files checked in, assets that move or are moved and then changed that git "loses track of" due to the significant change. This just may be me doing silly things like moving a file and then editing it and then committing it and git thinks I deleted it and created a new file.
Cursor Origin Code Hosting
When I've worked at big companies they had bespoke SCM that resolved this. But i'm curious about mid size teams that are big enough to overwhelm git but not so that they can rewrite it.
The problems start when there is a lot of changes on one branch and usually the first band-aid is to put a commit queue in place. The few very big companies that need even more than that probably aren't going to put their code on a new git hosting provider tomorrow.
> Those down-voting don't understand why any sacrifice of convenience or tech is worth it.
FWIW my comment was actually highly upvoted but it looks like X employees and/or Musk fans showed up and downvoted all criticism in this thread.