This was within a bank and the code in question was related to enabling Apple Pay from within the banking application. The consequences of that information and code leaking or being seen by anyone who had not signed the NDA were very serious (don't remember the details but it made the lawyers were extremely stressed about it).
Needing to figure out a way to protect those parts of the codebase it was decided in the end that the "easiest" way of doing this was to split the repository in half, with the actual artifact building taking place from the half that had the NDA code. The rest of the application (basically the whole application) was then used as a dependency by it.
Still didn't quite solve the issue, but access to that repository was heavily controlled.
Keys, secrets, etc. yes. But code? What am I missing here?
See also: the time that ATI's CEO told his employees that their chips would be powering Apple's to-be-announced hardware a few days before the announcement. Steve Jobs responded by pulling all of ATI's hardware from its demo units at the announcement, not mentioning ATI at all, cancelling a joint demonstration of the Radeon card that was going to be in the system, and never partnering with ATI again.
https://web.archive.org/web/20001216031800/https://www.zdnet...
> The incident began Monday when ATI, which supplies graphics cards for all Apple's current models, issued a four-paragraph news release that stated its Radeon processor would be featured in three new Mac models -- none of which were announced by Apple (Nasdaq: AAPL) until CEO Steve Jobs' Wednesday morning keynote address.
That's kinda a no-no for partnerships.
Except of course shipping ATI hardware for years afterwards, then also using nvidia, then dropping nvidia and only using ATI/AMD until transitioning to Apple Silicon.
You are looking at the problem from the wrong direction.
If you build a honeypot, to trap hackers, does it behove you to explain what the bait is, and how the trap works?
Know your customer, fraud detection heuristics, finger prints, behavioral triggers are all areas where banks, and financial institutions need to keep the sauce secret. Telling the other party "how" you catch them just gives them the steps of what not to do.
Does Gitlab do better with this?
Git enthusiast would often tell you to do this separately with a submodule, and set permission on the version control forge software level (which means Gitea/Github private RBAC access to certain repos for cloning), sure, but that is also painful as hell.
But my point is that all of this is exactly by design from Linus Torvalds's need for Linux Kernel to replace BitKeeper. Git simply isn't the tool for everything, it was developed for a software project with liberalism in mind, but corporate stuff is monoculture and prefers proprietary, shut-in model, and the eat your own dog food mindset, and no wonder it is so painful to deal with.
Unfortunately there are many people who think git is a panacea and is suitable for all version control tasks of anything.
When you use git submodules, you now have five or six problems.
If you need to NDA the core stuff instead and thus can't pull it as a submodule, the only thing I can think of is to pull the core as binary/compiled artifacts.
Git has no built in authentication or RBAC. Thats not what its for. Its flat file source control.
I swear loads of people havent a clue how git works or why it exists...most of the git based cloud services out there are 90% additional crap bolted on.
This is a weak argument you could use for any missing feature.
That is partially incorrect; you can restrict writes via hooks but not reads; you'd need a workaround like submodules
It’s fine for things that you want devs to be able to see without the Git host being able to see them, it’s less good at RBAC because there’s no real “identity” component at read-time.
If there is an asset that is updated very frequently in the early stage of development, you'll be charged for all the storage for the rest of the repo's life. That happens a lot in gamedev: most assets go back and forth early on but once it's done no one will touch them ever.
At my dayjob we used Git LFS for a bit, but foud it unworkably clunky - we eventually found it easier to just make a separate "LFS" repository and add it as a submodule to the main monorepo. Now we can rewrite the history of the LFS repo on an as-needed basis.
That's one of the features of Git LFS is separately managed storage and lifetime.
You are correct that GitHub does not offer that feature.
Git is bad at such volumes of binary assets, textures, models, sounds, etc.
This is used so e.g. an artist gets a repo that contains sources for the art assets, while a programmer gets the same repo but instead of art sources, it downloads the already produced binaries. As a SE, you just want to build the code, and don't care about 800 GB of art asset sources.
How well does Mercurial work in this situation (or even Subversion, given that Perforce is non-distributed like SVN)?
1. Code - Git
2. WIP art, shared assets (logos, marketing materials, etc) - Google Drive (because things are often changing, getting passed around, etc)
3. Finished assets (PSD files you're done with, or you think you're done with) - SVN (because we wanted a log of who contributed to what, wanted artists to be able to pick up where someone else left off; having a log of who made changes to a given PSD)
4. Assets rendered out to PNG to include in the app bundle/publish to the static file servers - Git (because those files never changed after being published so the git history wasn't polluted with unneeded files)
I've also used LFS, which is... a fine workaround, but still not great. Users who don't have it configured can still commit binary blobs; users who don't have it configured will clone files incorrectly; if the LFS server is slow, unavailable, unreliable, then the system starts to behave oddly; you need a Git server that supports it.
It was a huge hassle to manage; having a system like this would have been a godsend at that company, and if I still worked there I would be spending all day importing our codebase and assets into it to see how well it works.
SVN makes sense cuz it's done and dusted, but I could see the Drive gettin real messy real fast if things change a lot
In fairness, p4 predates most of the VCSes we consider "modern", so I empathize with a lot of the underlying architecture decisions. However, it has and continues to utterly fail at improving at a reasonable pace.
For example:
- p4 tracks file metadata of client workspaces on the server (sync'ed locally, opened for edit, file revision, etc) and uses this as the basis to avoid doing unneeded work. If this becomes desync'ed, a reconcile or force sync must be used. A reconcile can take hours, potentially days; it tries do detect file moves by default, so likely at least O(c^n) for some c>1. I have never personally seen a default reconcile operation _complete_ over any modestly large game code base, and in practice, people accumulate a litany of workarounds and scripts to fix this for themselves.
- Scripting p4 is a nightmare. Documentation is poor, schemas do not exist, and all the language-specific libraries are just thin wrappers over its C++ API.
- By default, p4 "helps" you with text files by "correcting" line endings on sync or even converting between encodings. This works until you have a mixed-OS environment, and discover a part of the pipechain that _must_ have a certain style. There are various levers to pull to make this better, but I've yet to find something fool proof.
- By default, p4 keeps flies read-only, only unlocking them when explicitly marked as being edited. This means, to avoid having to do this manually, every tool you use needs to be p4-aware. Or, you can turn this off, and choose to contend reconcile instead. (See above)
- Branching a modest game project, with, say, Unreal source code, can take hours. And this is the quick version where you ask the server to simply create new metadata, with no file transfer to a client.
- p4 is licensed by the user-account. Every user entity in p4 not intended exclusively for performing backups and maintenance operations counts toward this, including users required to integrate with other services. Plus, often times, these integration users must have admin access to be useful. The security posture is horrific.p4 also significantly predates VCSes we consider obsolete. p4 is almost a decade older than SVN.
This is why I wish more command line tools were split into a library that does most of the work and a cli module for purely user interaction. Parsing stdout seems so unnecessary and could be avoided if a program could simply import a library.
- The P4 cpp api was apparently designed before any modern Cpp std lib was available. And is at best archaic, and stringly to use.
- P4 encoding support is pain in the ass to configure. And ensist on adding or removing bom to files.
Git-annex ?
Well, it is intended to compete with Git for version control. It's just that Git happens to be so bad at some aspects of version control that it isn't used much in those cases.
There's no good reason that Git couldn't be good at versioning binary files, or splitting up large projects. I mean people have tried - there's LFS and submodules. It's just that those both suck balls.
I was kind of hoping Jujitsu or Pijul would take a stab at these major Git deficiencies but unfortunately it seems like they are content to do them as badly as Git does.
Git does very well at the 80% and with tools like custom merge tools and git lfs/annex and git sparse "cone" checkouts can get pretty close to hitting the 90 or 95% case.
But yeah, so many of those extra tools in that 80 to 90% area are awful to work with because they aren't the default, aren't out the box, are hard to configure and get right. Partly because it always seems like there will be a gap in that 95%-100% window and partly because the use cases that need that 80% to 90% often are only "just 10% of use cases".
(Which is also to say that to survive Jujitsu and Pijul and others seem to have to work to make sure they handle the 80% base case extremely well just to compete with git, they haven't necessarily time to think about the 90% or 100% problem.)
(ETA: And also relates to why game development seems to feel the 20% cases more, because by volume of data game development is certainly closer to a flip of the 80/20 sides with 80% or more large binaries by volume.)
If perforce is the best there is out there for large binary asset management, then there is a blue ocean worth of potential improvement for git.
Perforce is a piece of crap, a relic of the 20th century that must die in a fiery inferno.
The fact that it was easy to clone a subdirectory was nice; the fact that branches were just subdirectories also was not nice. The fact that tags were mutable since they were also just subdirectories... the fact that every operation you ever did required going to the server (commit, log, checkout, everything) made it a pain if you were on a slow link.
I can't count the number of times I was inspecting SVN history and had to just 'svn log > /tmp/svn.log' so I would have the whole log locally rather than having to hit the server each time I wanted to refine a grep.
SVN was actually quite decent for game development, definitely more robust and (non-technical-) user-friendly than git+lfs.
(and SVN isn't really compatible with the work-from-home era unfortunately, you really needed a big server on a gigabit LAN)
Thats not how everyone else works.
We're all using package managers to help with massive amounts of 3rd party dependancies (why are you version pinning in any place other than your repo, why arent you pulling updates through your repo and reviewing them)
We're reliant on tools like artifactory to make sure those depedancys dont disappear or are not corrupted.
We use yet other tools to manage our binary files (this tool would fix that).
Github, gittea, gitlab, bitbucket... have all added piles of tooling around git, that are grafted on around its short comings.
> It's crazy to me that nobody has made anything better yet.
Because our entire industry has fallen into the rut of "more tools", of stacking turtles (https://en.wikipedia.org/wiki/Turtles_all_the_way_down ) rather than fixing the real issues that hold us back.
> Although I could start that myself and yet have not.
Because unless your a Google or a Linus, no one is going to look twice at your tool for something that is this important. Im not even sure that epic games has the good will, or trust to launch this.
I am going to give them the benefit of the doubt and take a long hard look at it, but my optimism is tempered. But unless it offers a LOT more than git, the extra overhead (lacking IDE support, deployment changes and all the other tooling in GIT's orbit) it isnt going to be a worth while change.
This is why I'm not motivated to build something better. I don't think anyone would care.
He’s in a category of influencers who post constantly about gripes and grievances and smug superiority. Some people like that content but I can’t stand it.
I really like hearing about indie development and small teams, but you don’t have to present everything as condescending superiority over the industry. That’s not the part I find interesting.
His criticism isn't limited to AAA game studies. He's a vaccine skeptic and is pretty heavy into far-right influencer garbage. All very surprising if you only know him from his games.
This Tweet is a classic https://x.com/Jonathan_Blow/status/1854708962462982465
Jonathan Blow:
> It doesn't help that all males currently under the age of 40 were raised to be supercucks
I started following him for his gaming work and talks. His influencer content is something else.
I think being influential just does that to people, with high regularity.
Huge teams are more often than not the sign of bloat and inefficiences.
Now there's an argument to be made that many don't need the remaining ones but to claim that you 'replaced PowerPoint' for anyone but yourself is ridiculous.
They're good at demos, I give them that.
I'm wondering if you're confusing Windows terminal with cmd.exe?
Windows terminal is not the shell. It's a terminal emulator. You run a shell inside of the terminal, for example you can run... Powershell.
Command history is a feature of the shell.
https://epicgames.github.io/lore/explanation/system-design/#...
If this supports dynamic hydration of files, either as they're accessed (like Dropbox with offline files) or by somehow knowing which files need which other files (building a dependency graph) then it could be a massive win both for speed and efficiency of downloads but also for conserving disk space on developer machines.
And since it has API bindings, it's possible that's something that could be built into IDE plugins, so that your editor (Godot, Unity, etc) can know which assets need which other assets and automatically trigger hydration, including when you e.g. try to use a new model/texture/etc in a scene that hasn't used it yet.
Also git-lfs is a crutch that breaks more often than it works :/
(I agree though that for small game projects, git is mostly 'good enough', even without lfs).
For the longest time Git tools were really poor. In recent years there's a few ok ones, like Git Fork, though I wouldn't know if those tools scale to the level of a AAA team size repo and not fall over.
At Fortinet we migrated our SVN repositories to git and ran into a ton of issues; developers over the past ten years had done tons of little mistakes that added up, like accidentally checking an entire Windows virtual machine into the repo. In SVN they deleted it and no one ended up caring, but in Git of course it became part of the repo history.
I did a huge amount of work for the migration, 99% of which was analyzing each repo to find out what files/file extensions were overly large, and then either:
1. Filtering them out of the git history completely during import
2. Converting them to LFS objects after the import
The LFS process was certainly better than the other alternatives, which were 'check everything into the git history' or 'remove all the un-diffable binary files and hope that they weren't needed for anything', but it was still not ideal.
Every developer (out of thousands, across multiple countries, timezones, and native languages) had to set their system up properly; if you missed a command, or if you reinstalled your OS and forgot to set up one of the aliases or hooks, then you would end up checking binary blobs into git rather than LFS, or checking out LFS idents rather than the actual files they needed.
We also had the issue of developers fetching code over SSH but LFS files over HTTPS, which would be fine except that we wanted to prevent access to HTTPS from most subnets, so while the developers could use SSH to clone or pull using their 2FA token their client would then make an HTTP request that wouldn't work unless they were on the version control VPN, which.... blah blah blah.
So yeah, it worked better than the alternative, but it did not work _well_ a lot of the time.
it's also tough when you have 1TB of data, over 1mm files and you might want to lock hundreds files in one go
I think branching is the one thing that didn't get more complicated with LFS.