upvote
I’m not even sure I agree it’s impossible for one person to understand a large codebase. It’ll depend on size of course but plenty of major pieces of software have been developed by a single person capable of understanding the whole codebase.
reply
I've developed plenty of large software projects, and what happens in practice is that you build up the context for the part of the project you're working on when you make changes. If you could just hold the entirety of the project in your head we wouldn't need tools like debuggers or test harnesses at all. You'd just know that all the changes you made are correct. That's obviously not how things work in the real world.
reply
Author here. Pointing out that we can't hold entire projects in our heads is not a made up excuse. And doing things like breaking things up into modular components that have specifications and tests is precisely how we combat cognitive complexity. That's exactly the approach I am taking with Jolt where there is a large specification and conformance harness used to ensure semantic correctness of the language. https://github.com/jolt-lang/jolt/tree/main/test/conformance

Meanwhile, anybody who's maintained a large scale software project knows that even with the best practices, problems can still be introduced into the codebase. Nobody is arguing against limiting context or doing as much as possible to ensure the contribution is doing the right thing. Yet, the idea that you can have full understanding of how the feature will interact with every edge case is frankly laughable.

The notion that you can architect a perfect codebase where context and scope of the effect of any change is always completely clear is idealistic in the extreme. Of course, we should strive to keep the code quality as high as possible, nobody is arguing against that. The reality is that large projects are complex, and nobody has come up with a way to write code in the way where it's possible to perfectly control the scope of the effects of any one change. Every single large scale project out there has bugs, and regressions no matter how much the authors strive to make it perfect.

reply
> Yet, the idea that you can have full understanding of how the feature will interact with every edge case is frankly laughable.

That's not what I'm saying. What I'm saying is that the contributor should make a reasonable attempt at producing a bug free patch. And it's up to the project's leaders to make it easy to do so. And one of the easiest way for the latter to do so is producing design docs and isolating modules as much as possible so that it's efficient to get the needed understanding.

So both parties each have its own responsibilities into making quality contributions happen. I'm not against LLM usage. What I'm against is making the project difficult for human understanding because human understanding is limited or whatever. That's why I point out the README and the Makefile.

reply
Nowhere have I suggested that the contributor should not make a reasonable attempt at producing a bug free patch. And Jolt provides plenty of design docs actually: https://jolt-lang.net/docs/rfc/README.html

I also strive to have a clean and modular codebase, including building tools specifically aimed at helping ensure code quality in addition to manual reviews https://github.com/jlt-commons/jolt-metrics

So, if you'd like to point out specific gaps in documentation or problems in the code architecture, then feel free to share them and we can discuss that.

> What I'm against is making the project difficult for human understanding because human understanding is limited or whatever.

I'm against that as well, so not sure what your point of contention is really. The README simply points out that there are limits to what we can reasonably understand, and that the mitigation for that lies in the process such as having good specifications, benchmarks, regression tests, and so on.

Again, if you can point to specific aspects of the project that you claim make it difficult for humans to understand, then please be specific regarding what they are instead of making vague accusations. Also, it's quite interesting that you chose to take a few bits out of context while ignoring the following which is directly relevant to the accusations you are levelling here:

> The core skill in using these tools is the same as the skill needed when writing code manually. One has to understand the given requirement, the shape of the solution, and how to verify that the code produced actually matches the intent.

> What these tools do is automate the mechanical aspect of producing the code. Once it is written, you can read it, understand it, and change it as you would with any other code. The intention and the ideas come from the human using the tool, and new ways to automate the work of typing out code do not change the role or the responsibility of the developer.

> Focusing on how the code was produced rather than on the code itself dilutes the responsibility since blame should never be deflected to LLMs. The ownership of the code rests with the person submitting the code, and they must take responsibility for its quality and correctness. Either the code is clean and lives up to the standards of the project, or it will be rejected and the author will be asked to make changes to address the problems.

Can you point to where in your mind I'm suggesting that developers shouldn't try to ensure code quality or understanding?

reply
I'm using jolt in a new project and yesterday submitted my first PR to jolt, to improve some testing functionality. I'm pretty confident in it's quality and I'm super excited about jolt. It has opened very interesting frontiers for me as a clojure developer.
reply
Thanks for the PR, and it's really exciting to see that Jolt is becoming useful for people other than myself now. :)
reply
> I get the impression that you must be new in your programming career, and hold idealistic views of the reality of what maintaining large projects looks like in practice.

Personal attacks for someone pointing out some issues with the LLM apologetics. Nice.

disclaimer: 27 year professional in this field and I mostly agree with OP.

reply
Anyone who's seen the author communicate here and on Lobsters will know that this particular critique irks them and they tend to react exactly like this when it comes up. It's a sore spot, and one the author should probably try to work through on their own since they seem to spend a lot of time vibecoding things.
reply
If you have 27 years of professional experience in this field and still think that you can have total knowledge of how a piece of functionality interacts with the overall codebase in a large software project, then I really don't know what to tell you. If that was the case we'd be writing perfect software that never has bugs.

Also, I have no idea what any of this has to do with LLMs. Nothing discussed here was LLM related in any way. Nor am I doing any apologetics for anything here. I'm stating basic facts that anybody with even a modicum of intellectual integrity would acknowledge.

But if you have any actual counterpoints to what I said then feel free to share them.

reply
Don't let the anti-AI people bug you. They seem to pounce on every thread and complain about LLM usage to the point where it drowns out any legitimate discussion.

I'm a heavy LLM user myself and it's reinvigorated my passion for coding again. I'm making things I never would have otherwise made. I probably wouldn't even be writing code anymore. I've worked on large codebases in my career and I felt like I understood them well. Still, it was so time consuming to get to that point in the first place and I would still have to familiarize myself all over again when working on components that didn't get touched often. An LLM makes it so much easier.

Ultimately, the developer has to own the code and be accountable for it. As long as that's true, I don't see what difference it makes what tools the developer uses. It's true that you can shoot yourself in the foot with an LLM, but you're probably going to shoot yourself in the foot without one too.

Anyway, cool project. Thanks for sharing it.

reply
Thanks, and completely agree with all that. If we treat LLMs as a simply a new tool that developers use, then nothing has really changed. The person working on the project is still responsible for all the decisions, code quality, and the direction of the project.

And these tools are fascinating to me because it's a completely new way to develop software. I've been writing code for over two decades now, and most code you end up writing isn't interesting or novel in any way. Any app will need to talk to a database, have some services, have a UI, and so on. Once you've built enough of these things, you're just repeating the same patterns over and over.

LLMs are something that's genuinely new and different. Nobody really knows how to get the most out of them. Everybody is still figuring out how they work, what they can do, what workflows to use around them. And that's really exciting because you can discover some new trick that nobody's tried before. So, I find this reinvigorating as well because it brings back the novelty and excitement that I remember having when I initially got into programming.

reply