I have had experiences out of my technical depth where I’ve instructed an agent to do something technically nonsensical. The agent tried to do it, creating a mess. Nothing works, I get frustrated, only to realize I have a fundamental misunderstanding of something technical.
A similar thing happens when you ask for something, but the complexity for the perfect solution is 10 x the complexity of a simpler, 90% solution. And sometimes you only realize that looking at code (or feeling lots of brittleness)
You're trying to middle road it. He was talking about actually treating it as a black box, so just give the model your expectations and goals it needs to meet, not tell it a rough solution for it to apply in detail.
The expectation/goal approach in a ai edited project works better then the middle road since opus5.5 ime. Depending on your project structure, it may not be viable however without large refactorings
Other times it will one shot things near perfectly that I was expecting to just use as a reference. Demo pages for work I've done its so fast and gets good enough right out the door
Every time I've tried to use it to write something small in my compiler? :"( pain. It's the perfect intersection of looks good and covers 90% of cases fine but explodes far away because some subtle invariant is wrong. I can't always think of the invariants up front without writing the code myself
But to preempt a "I can't do that in my brownfield distributed monolith I'm maintaining at $dayjob": yes, that's why I said it may not be possible without refactoring.
My point is that you can ask for unrealistic outputs even if you’re treating the system as a black box. This is the classic case of higher level managers asking for infeasible results from their engineering team because they haven’t considered they’re trying to solve the wrong problem. Depends what you’re doing but I don’t think agents tend to push back on badly specified goals unless you’re proactive about it
For a different perspective, the european peers in my developer network, while never outspoken against their company, like, and redistribute content that circle around the same points. Some have fixed mandates to use AI, in order to use AI, working more to keep the product stable (even outside normal work hours), or complains about the divide between management and developer's perspective on these tools.
There is also some consideration about compliance issues and liability when you're stamping commits with your name (luckily for almost all software enginners there is no malpraxis law to punish gross negligence).
For context, I work in a big tech company with virtually unlimited tokens on the best models, and a lot of tools integrated in our workflows. The culture makes it impossible to go against the wind. Even if you don't want to use AI as much as others, you won't be able to prevent them to do so.
But what happens when your Node runtime, OS, compilers, or even libraries start regressing to the same lower-quality level? Everything starts collapsing and falling apart. That matters.
For the last 20 years of my career, I have prided myself on keeping a clean codebase, making 'art-not-science' decisions of when and what to abstract, scope boundaries, then implementing them cleanly, specifically to be easily grok'd by other developers. But after building, from scratch, and operating an (essentially) vibe coded software solution (in typically a very engineer led/low level space) over the last 9 months, I am still able to move at super-human speeds in terms of delivering features, investigating and fixing any issues (there aren't many, certainly less than most human teams I've been in) - whether that's because of AI advances or just their innate quality of being able to tackle the dross coding tasks that always got avoided by humans in my past I'm not sure but I'll stick with it until a better approach becomes apparent!
I feel my specialist skills (coding, debugging, devops) has been devalued - I can't justify holding those as my value-add proposition any more - but new skills are replacing those (context/AI/agent engineering) in equal measure.
And you’re right, with a bit of direction and further prompting it’s quite easy to get your code in a reasonable shape. So that is what I do now: the agent does all the work but I still whip the output in shape.
Mostly my hope is that at some point soon the agents will start writing clean code. But then again, maybe clean code for a galaxy brained agent is not the most optimal code from a all-in token efficiency perspective.
That’s more of an artist thinking than an engineering thinking, favoriting elegance (cleanness, readability) over practicality. This was not really valuable even though you prided yourself on doing so. It’s like endlessly gushing on the design aspect of the report (the medium) instead of just providing a well-written one (the message).
Code is a medium, and being an artist about it can feel good, but the information processing mechanism (the message), is what’s important. If, after using AI, you can’t reason about the mechanism (nothing about code, files,…), you can’t essentially justify anything about what happens in the software. Meaning data given to it has a non-zero chance of giving garbage answer.
> but new skills are replacing those (context/AI/agent engineering) in equal measure
This is still highlighting surface activities, not anything pertaining to engineering.
No one is going to want "organic software" for their Node runtime, their web browser, or their OS. Some things just need to work, and work efficiently, 99.999999% of the time.
Depends on use case
You definitely can use those things to improve both speed and code quality at once. But then your speedup is while quite noticeable not astronomical of course. Also the work isn't easier (apart from less typing), just different.
Doesn't seem like a good strategy to me.
Using AI is one element for a pipeline that produces code.
If you architect well what you generate then it stops being slop and starts behaving like a cohesive system where code is produced automatically.
So, no, you will never be able to produce code automatically. You can and certainly should use AI as a code assistant to write code faster, but the human should always 1) make all important architectural decisions and 2) carefully review the outputted code
The human bottleneck is necessary.
So the system accounts for mistakes and has protocols for removing bad modules/code.
If part of the system go down the rest will remain functional
Edit: and yes audits are part of the pipeline
This obviously won't fly for "serious" software such as airplane software, but I'm willing to guess that even customers of a low-stakes SaaS would get pissed off if your product regularly turns up new bugs.
I think it's easier to just do code reviews...
Both me and LLMs are constantly auditing software.
In-fact auditing and QA is mainly what I do now and human audits become much easier when it's scoped to modules.
You cannot do that with humans either.
The types of mistakes LLMs will make are failures of alignment, that is, basically unpredictable. They could fuck up anything. Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.
It will happen less and less as they get better and better, but as the volume of software output keeps growing, it will happen with a consistent defect rate. And we won't be able to catch it prior to shipping, because humans won't be able to review all the code or verify all the tests.
I don't think anyone's started taking this seriously yet.
Gen Z has learned that usually, your dedication to work is not tied to the monetary reward.
> Where a human fucks up by introducing a crash, the LLM fucks up by just not solving the problem you asked it to solve.
Have you ever done a group work project at school?
the problem is that the "architect well" part is also being left to the LLMs
That's exactly the strategy that most companies have adopted by now.
Once you go 100% AI code generation, there's no going back.
Picture a foolish person falling out of a hot air balloon.
"Gosh," they think, plummeting, "the balloon is going upwards SO fast!"
“it would have taken me days/weeks/years/… to learn and produce something by myself”.
reminds me of all the PHP spaghetti code that would float around because it was "quick and easy" -- eventually that all becomes brittle and someone has to come in and rearchitect it from scratch
you may not pay the price, but your successor will
but then again, you could say that's your employer's problem, not yours (and you'd be right)
the biggest problem is the future, not the present. Presently, we -- you, I, others -- still have the skill and experience to look inside the box and understand it, guide the LLM to make smarter decisions, etc.
not only will those skills atrophy in ourselves, but the younger generation won't even develop them in the first place. eventually you have almost no one who understands anything about how things work, they just know how to issue high level commands. It's like if eventually everyone working at CERN just knows how to push the buttons on the LHC but doesn't understand much beyond that. That's where we are heading.
This raises alarm bells. It's a deeply unserious take on software development that I hope doesn't spread (but I know will). YOLO it all.
We absolutely will regret the slop AI, as it will become load-bearing (ahem) in short order. There will be critical outages and people will die.