"Couldn't this be stateless?" "Do you have a plan to be able to shard this?" - We will almost always pivot from the agents initial design but the agent is able to easily understand the reasons and benefits and align quickly.
If I was writing the code myself, I'd often have to make compromises between the ideal architecture and the level of effort required to implement it - now I can just always have the ideal architecture.
Now you have what you think is the ideal architecture. Since you didn't implement it, you didn't discover it was not the ideal one mid way into the implementation.
May be it is too complicated, but you wouldn't know, because LLM is doing the implementation. If you did implement it yourself, you might have spotted a critical point that might simplify the whole thing...
This is the "Jodie Foster in Contact listening for the SETI signal with headphones" theory of how production software systems work.
But then, one small detail changed everything. And it wasn't something I'd have thought of until I was writing the code.
I shudder thinking about the number of these issues hiding in LLM-generated code.
There is a difference between LLM-generated code that somebody merged essentially unseen and LLM-generated code which a human then looked at carefully and massaged until they were happy with it.
The first version can hide a lot of crap. All you see is the end-to-end functionality and there can be corner cases or scenarios which you have not tried and in which it is terrible or plain incorrect.
The second version hides just as many issues as human written code.
Unfortunately, the debate that our field has about LLM-generated code mixes both quite freely even though they are quite distinct.
I feel the opposite, like if I go to an agent without first knowing what I want to build, I'll never figure out what I'm doing or why and it'll run away from me.
I pretty much always go back and forth and have the agent write out a plan to a file and review it myself in my text editor. I still sometimes end up with surprises I disagree with, but I don't really find it to be true that the LLM ends up trying to "drive all of the thinking".
When I'm thinking through an architecture, I not only instruct it to refrain from writing any code, I don't even necessarily tell the agent what I'm trying to build.
I walk to work and home with ChatGPT Voice and AirPods. I ask it to be Socratic and I just start rambling the top of thing on my mind. After 20 mins of back-and-forth it's usually teased an answer out of me or I've teased an answer out of it.
And do you feel comfortable with the setting that a corporation has a detailed log of your deepest thoughts?
I start talking to it while doing menial tasks like cleaning or doing the laundry, and I discuss the architectual decisions and options until I come to some resemblance of a plan.
Good side of this approach is that I can't just "skim over" or "copy paste" things - either I understood them and can repeat them myself, or I can't. It takes more time than /grill-me and similar approaches, but it's the only approach that doesn't make me want to claw my brain out.
It's very enlightening, though it requires being comfortable with being alone with ones thoughts. It seems to me many people are not and need constant distractions and/or dopamine kicks.
Other times, I let the agent create a black box with a well-defined interface contract. I don't care about the architecture inside the black box.
The agent can rewrite half your codebase in one day, so architecture stops mattering for the most part.
But if the codebase is large enough, and if you don't have tests for everything, you'll likely lose stuff along the way. So you'll then spend at least another day tidying up the rewrite, just getting it back to where you were two days ago.
Does that sound like fun? Wouldn't it be easier to just plan it correctly the first time?
Some apps don't care about scalability and performance but many do. Ignoring architecture all but guarantees inefficient, wasteful software.
This is not true for everybody or for every project. Sometimes it’s about the code and not about what it does. Sometimes it’s about learning. Sometimes it’s about the fun of creating. “Only the end product matters” is a narrow view of the world of software development.
For example is unknowingly writing a security flaw ethical, when you could have used a set of processes to reduce them before release that would have make the entire thing take longer and cost more. Seems like programmers need a lot more ethics classes as ethics are part of any large scale process.
Besides, ethics come from upbringing and social influences, not from attending a mandatory ethics class.
LLMs benefit from abstractions for the same reasons that humans do. More information in the same amount of text. Fewer working parts to juggle so fewer ways to make mistakes.
Similar to the output of a compiler. Nobody (with very few exceptions) reviews its machine code output. No reason to do that. If you want to change it, just recompile.
(I don't share this view, but I think a substantial and growing fraction of folks does.)
I'll create a simple "framework" of what I know works. After that's there the LLM is fantastic.
But why would you care when an AI can just rewrite it? Yes, but can it rewrite it to a good architecture? Or just to a different one?
Does a good architecture make code easier for an AI to maintain? I don't know, but I think it's at least not proven that it doesn't.
Architecture has remained so far to be one of those domains where skillset dwarfs everything by comparison.