upvote
Right, but this is the most hopeful imaginable prediction: AI accelerates everything, but everything good is still mediated through taste and expertise, so being a (say) skilled systems engineer is important and rewarding, just with all the tedium distilled out of it.

There's a less hopeful outcome, where no skill is needed at all, because every conceivable thing is one-shot-able from a SOTA model. But then, at that point, you're saying something pretty profound about computer science (roughly: that there was no meaningfully more sophisticated and complex place to go from where we are today).

reply
My wife is not a developer at all but is highly analytical. I find her a better software architect than most developers I met.

I suspect there are a lot of people really good at “systems” - there are a lot of really clever folks out there - that did not go through the proper CS rituals and are thus shunned but in fact can take over your job in no time especially with AI.

My wife regularly gets pushed away by “IT” at her work and unlike others she pushes back, but they don’t like her. They start talking about db clustering details etc as if that has any bearing on anything. I am a dev, I know she doesn’t know what logical replication is, but I do know she can replace their team. She just needs one technical person to keep an eye on things.

I suspect we will end up with just a handful of building blocks like “this is how data is siloed, secured etc”, “this is how dashboards are made here”, “this is how email is handled”, etc and employees with AI just develop their own stuff on top of these. The “devs” just oversee the building blocks and the overall soundness of the flow. Don’t know how many you need once that is standardized..

reply
We're going through a very unique, fast-paced transition and the problem is mostly cultural/psychological IMO

People react before they've had the time to really process it, and show a lot about their existing thought processes

Many technical people IMO fetishize technology and processes, disagreeable-but-right authority justified by their willingness to work on things that most people find hard or just don't want to do. Computers can be a vessel for identity and self-worth - it's alluring to think of technology (and ourselves) as intrinsically valuable. A surprisingly large contingent seems to see work as necessarily zero-sum and adversarial, and make it a core part of their identity (and often not even particularly abnormally powerful or better for it) - like what even is the archetype there?

People like your wife are the ones writing crazy excel spreadsheets and VB6 macros, and making decisions about enterprise IT products, and solving problems with computers! They're actually probably more knowledgeable about "what is a useful and valuable way to use computers" than the ones who find computers fun and freeing and Just Want to Code.

Algorithms will dole out as much outrage as you can take but at the end of the day, doing stuff with computers that people want to pay for - usually, automating or securing things - is why people pay us!

One framing is people overindex on Being a certain kind of person or (the archetype they aspire to or identify with, that is valuable in a passive sense), over Becoming or Doing what's necessary for their work to be valuable (actively helping people, and stuff that's not fun). Very normal to feel grief and not want life to feel so uncertain, but it was literally always uncertain! Life isn't over because the business people at your IT job want to build software!

reply
That’s something that I have been thinking about for a while now. Even prior to LLMs, many large companies have essentially been iterating on the same half a dozen ideas over and over again and there doesn’t seem to be much new coming out even post LLMs.

The most recent new things that LLMs have unlocked (regarding software dev) for us is reverse engineering and combining old games, unearthing decade old vulnerabilities, and helping root and debloat consumer hardware. Which are all cool and useful, but none of it being a “next great thing”.

Perhaps LLMs are really doing is highlighting how the software industry was already nearing its economic maturity and this just shifted the timeline from 2040 back to 2027-2030.

reply
I have yet to see LLMs do well “autonomously” on anything that doesn’t fall into any of these 3 buckets:

1. well specified

2. well tested (which is a form of spec)

3. represented in the training set (i.e. not built on a corpus of human developed examples)

For example, a lot of the compiler porting efforts fall into all 3 buckets: compilers are well specified (there is a set of valid inputs and an expected behavior), they are typically well tested, and as a bonus, in a way that is implementation agnostic thanks to the clean interface (think: integration vs unit tests), and are very well represented in the training set.

reply
You can "solve" 1 and 2 by N-version programming. Have the same code implemented twice, or more times, you get free error testing by fuzzing or running replicas in shadow mode. In order to make replicas maximally different choose a different language, framework, agent harness and LLM.

There remain some nitty details related to tracking randomness and choosing what interfaces to log. But even having your code testable by replicas would help organize it better and expose different assumptions made by the language or library stack.

When you get different outputs from replicas it can be 1. one of them has a bug, or 2. the spec is unclear -> fix the bug or the spec. So this method catches both sides. It also might be faster than traditional code based testing, you can build replicas in parallel. A replica is an open ended test, unlike coded tests it can be tried on open ended input values.

The core idea is that it is easier to implement twice than to imagine bugs. We already solved implementation, but bug finding means running code in your head, it touches the halting problem issues - can we really imagine behavior from code? I don't think we can do it well. That is why we should just make replicas instead, tokens are getting cheaper and cheaper by the month.

reply
I had an LLM tell me that I mattered once, which I know for objective fact is not represented in the training data.
reply
it is and you do matter
reply
and yet some OSS (projects not really known for 1 and 2) multiplied change velocity the last year, implementing features via what is essentially vibecoding.

If you look at that, this approach is producing a pile of slop, where one LLM implements a feature, two weeks later another LLM spots an edge case and without much human review just crams in the code to fix that. Even at 1-2 months of that, those repos have significant changes and the commit messages get weird. I wonder what happens if the LLM won't spit out a fix anymore?

Will the human maintainers who started this process because noone thanked them and noone wanted to work with them on their bright new ideas just reset to the last commit they understand? Will they drop the project? The sunk cost for them is enormous. The time-effort required to fix the mess is gargantuan already. To me it increasingly seems that after the LLM industry robbed most humans of their collective labour output it will now rob them of labour aka time as well. Some people (like you and me) seem to notice that but most just go on and essentially spend away their life on machines now. I guess the problem is that most of these people have been severely cut off from using their own agency.

reply
Don’t forget we don’t live in a pacifist utopia. Companies don’t work on the same ideas just because they think they can make it better.

Companies work on the same ideas because they want to own IP, data etc. and then charge money.

The other way around they work on the same ideas not to be charged by others and control their business.

“software industry” is not some big happy hippie family. Patent wars, browser wars, OS wars - well we get a lot of open source for free but that’s not the norm.

reply
Honestly I think you're on to something, as by far the most novel and interesting use cases I've seen for LLMs in general are mostly around reverse engineering things, especially if you can hit it with Ghidra, but honestly anything that can feed some sort of data and control back to a model.
reply
> the software industry was already nearing its economic maturity

What I'm sensing is something I might describe as the Oligarchy reaching the point of significant societal pushback, like the labor rights movements that ended 80 work weeks, unsafe conditions, and child labor in factories (i.e. The Jungle).

My hope is that we can use this moment in human history to break the user tracking and targeted advertising industry. It's been so harmful to humans, but in doing so, we will likely break a major revenue source that supports our well above average salaries. I still think it should be done.

reply
What indications do you see of significant and effective societal pushback?

I would love to believe this but I don't see it yet. I see a lot of anger in some circles, but I also see the machine marching on as normal.

reply
What would be funny with LLMs, having been purposefully trained to be the best software creating things on earth, wind up inadvertently making software virtually valueless (because it can be made and copied so easily) and undermining the value of the companies that proliferated their use in the first place.
reply
> What would be funny with LLMs, having been purposefully trained to be the best software creating things on earth, wind up inadvertently making software virtually valueless

This is the world the accelerationists are trying to create, is it not? Fully automated luxury space communism through AI solving all earthly problems.

We're just unlucky that software is one of the first moats to fall (alongside the humanities), because it doesn't require developing novel ways for the AI to interact with the real world...

reply
We'll have to separate the hardware providers from the software providers, which I'm growing more fond of. The US Ai stack is largely a vertically integrated cartel.
reply
> but everything good is still mediated through taste and expertise

I'm reminded of https://archive.ph/a9Cli (ns;nt)

reply
I haven't yet seen any significant products made with AI that are at least used long-term. All these browser demos, from the Adobe killers to Star Citizen, are just bells and whistles that I close after 30 seconds of use.
reply
I enjoy this quote from Antoine de Saint-Exupéry:

"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."

And I feel that sums up my feelings on AI this and that. Just as the wood and clay contain the sculpture within and it's the artist's duty and privilege to reveal that. So, is elegant software exposed from the conceptual mass, rather than built up.

The AI zeitgeist currently is 'how many features can I add' because we've tied self worth to work output instead of some other more personal or human rubric. AI will happily pile more and more and more code on your plate because it's a buffet. It's gone through reinforcement cycles guiding it to add whatever feature you ask. There is no training for elegance or careful considered design. Nobody is teaching this generation of LLM restraint.

The safest code is that which never runs. The most elegant feature is that which is not necessary.

reply
"The safest code is that which never runs. The most elegant feature is that which is not necessary."

I assume you run GNU Hurd as your OS? ...

In general I very much agree to the quote of the Little Prince's creator, but I am not sure if we mean the same by it, because you seem to value code elegance more than adding useful features?

Now "useful" does the heavy lifting here, but my point is a useful feature is useful, even if it is achieved by the most ugly unmaintainable code. And what counts as a useful feature will depend on who you ask, but in general you should ask some users. (They don't care how the code looks)

And sure, they do not enjoy a pile of garbage of new features that only work sometimes, but managed to destroy their existing flow. But "Nobody is teaching this generation of LLM restraint" this is hyperbole and up to each developer. There is no need to let the AI run on auto for hours, this is only one way how to use that tool. The other way is, you make the design yourself - and let the AI implement it.

reply
> The safest code is that which never runs. The most elegant feature is that which is not necessary.

Yes, but the most useless software is that which doesn't exist.

At some point, you actually do have to write some code and add some features, or else the most satisfied client is the one who hired your competitor.

reply
Claude Code?
reply
There have been a few vibecoded projects I've considered hosting on my home server, but it seems as if each of them doesn't understand when to stop adding new features.

They keep adding features that are outside the scope of the original project, without refining or polishing the original concept.

reply
and sadly that happens to a lot of existing opensource too...
reply
Agreed. Been experiencing this firsthand with an app I've been building myself. Vibecoded a working prototype ridiculously fast but then came the optimization issues. Since mine is very hardware dependent every different brand, oem stack, processor, video card, and every little thing changes how my app functions and a lot of it isnt even documented properly. Now AI is still very useful here, but you actually need to know what even to ask and fix in the first place
reply
My interpretation of "be more ambitious" is not only give it harder problems, but be more creative about what you ask. The problem is that most of the MVPs that people ask for a boring and lame. AI can help us make things that blow your mind, things that we would never build because they don't make sense, but the cost models for code generation have changed.
reply
> AI can take us from 0 to MVP in a prompt.

If you're being ambitious enough, you shouldn't be able to there in one prompt. Ambitious work with AI is done 24/7 for months and guiding it continuously, because it's not good enough to work autonomously for that long without going off the rails.

reply
[flagged]
reply