The earliest used models in this category would easily 10x (what did I just say) the creation of one-off short scripts, but you're not writing a 30-year app out of just a bunch of short scripts.
The METR time horizons graph suggests we can now get 10x (ahem) speedup on solving most coding problems that take us a few hours and about half of problems that take 2 days. Amdahl's law bites: even infinity speedup on half your problems is only 2x overall.
If you let an LLM loose, with a huge budget, what's the biggest artefact it can make before it drowns under the weight of bad decisions? The C compiler and web browser headlines a while back? The maths papers we see now that solve problems which stumped the maths world for decades?
But this is the other side of the same coin: teamwork. One person getting a thing made in twelve months vs a team of a hundred, you can scale up fast with money when you have a proof of concept, and an LLM can make a lot of proofs of a lot of concepts even in free accounts.
The actual hard part is getting it into idiomatic, safe rust, and I don't believe the LLM port makes the full transition any easier than doing things the old fashioned way: a dual lang code base like Linux.
Doesn't matter.
I do agree that the port would've taken a lot longer without LLMs though.
After an aquisition earlier this year I got the task of doing an SAP-Integration for the new company, last time I did this 5 years ago it was a 6 month task, but with the experience and skills ive gained since I estimated it would be a 3 month project (with or without AI, most work is just logistics, AI cant help much there).
In those 3 months I was able to not only integrate SAP but also deliver a completely modernised user-facing software for that integration. While I could have written that software myself in a vacuum it would have never been worth it financially, since it would have delayed the launch of the integration by 6+ months. Building the software post-launch of the integration would have easily taken 2.5 years at minimum.
But this is also basically a "spherical cow in a vacuum" scenario, where I was essentially acting as a solo dev, in full operational control of the project, with deep domain knowledge of the topic and an allready fully set up codebase that I knew perfectly while working down ideas I've had in my backlog for 5+ years.
What you have said is correct, it lets you build software much faster. The question however is: is that software making money for the company? (Not talking about what you built but in general)
I think, with AI, companies are saying yes to a lot of things they would have said No to ik say 2020. And as a result realizing “just building it” is not the answer.
Previously your GTM team or Product team would say “If we ship some big project X, we unlock $Y in revenue” but now people are realizing that those projections were really more of a hope. So companies are spending so much more tokens and shipping so many more PRs based on hope but a lot of it just doesn’t turn into meaningful revenue, especially not in short term
This sort of system only works with internal software and an unusual amount of data. If we were in the business of selling that software we could not have charged a higher price for the new version over the old, the tweak could only be unlocked because we were able to control staff hiring and staff onboarding fully to make use of the new changes.
As part of the aquisition I got access to their previous codebase which was some sort of incomprehensible PHP monolith, with the persons who wrote that code long gone. Thanks to LLMs I was actually able to extract the core useful concepts (again, sufficiently deep domain knowledge that I knew exactly what to look for). Without LLMs i would have probably extracted the absolute minimum and let the rest rot.
There is no reason a dev of comparable skill and domain knowledge would not be able to do that for what I built here.
By that point it might no longer matter though, but I suspect that such code would have a lot more exposed edge cases than one where someone actually thought things through before coding.
So yes, many aspects of my job are now 10x as productive, but turns out that improves my overall throughput only very little.
Second, the Internet didn't show up much in GDP and similar measures either!
But your point stands. Where are the amazing digital products/stuff? I get that it might take time to arrive as we scale up compute and learn new paradigms. But so much infra already exists (deployment pipeliens, everyone reachable on a smartphone) that we should be seeing something.
I'm definitely seeing indie-sized games that appear to have had significant input from AI, though I'm not sure the balance between AI for coding and AI for assets. My experience attempting this directly suggests that the current level they work at can make very simple games as one-shots, but anything more than trivial will produce outputs only as good as the developer's combined willingness to put in effort tweaking things and taking it all one step at a time, and their taste about what "good" even is.
I'm using spare credits to build and improve an isochrone map renderer, which I otherwise wouldn't have had time for (apart from anything else, I'd have had to become skilled in JS+wasm, somewhat of a pivot from iOS). This also requires taking it all one step at a time, having UX and UI taste.
Having lived through GeoCities since before it was bought by Yahoo!, taste is… well. Most people make things that nobody else actually wants.
But I am just as amazed with how little real life consequence it seems to have! Even software houses were hit more by interest rates than by this magical revolution.
If I couldn't directly observe Fable in action, I wouldn't believe in AI.
>> What's moving the goalposts? I am very much amazed at what Opus 4.8 can do. I push its code straight to prod.
>>> What's moving the goalposts? I am very much amazed at what Opus 4.6 can do. I push its code straight to prod.
>>>> What's moving the goalposts? I am very much amazed at what GPT5 can do. I push its code straight to prod.
>>>>>> What's moving the goalposts? I am very much amazed at what Opus 3.5 can do. I push its code straight to prod.
I've done amounts of refactoring and fixes and written tooling that just wouldn't have happened before.
I'm not sure what amazing new stuff y'all expect but the amount of technical debt in my projects is actually going down, cause I can finally get good enough test coverage, including E2E/load tests that actually prove whether the software works and scales or doesn't - just last week I diagnosed issues with SeaweedFS failing under concurrent writes when backing Sentry and could swap it out for Garage in a day, caught by a monitoring tool I slopped together that integrates with the Sentry API, no issues since.
The environment around me has gone from drowning in tech/ops debt to sort of swimming and at least holding above water for now (cause nobody will pay for 5x more tokens).
It's also insanely good for prototyping and being able to actually explore various ideas and shoot the bad ones down quickly instead of handwaving and looking at a loaded calendar, alongside being able to address well bounded tasks in parallel, better than human developers can - like I can give 5 GitHub issues to the slop machine and have it fix all of the annoying bugs. Issue with how some data shows up? Just feed it the DB dump and let it find out what's up.
Some projects have gone from around 500 code tests to around 4000, and before anyone says they're meaningless, at least 5% of those have caught real issues and helped a bunch, alongside linters and other tooling (including some tools I wrote myself). I've also written both native utilities and some web platforms for myself, side projects that I never would have gotten around to.
I'm measurably more productive than I've ever been (since I did measure that, looking at my commits over the last 2 years) but also burnt out. Still, it's the kind of burnout that's the consequence of context switching and lots of work, rather than the kind that I had years ago, where I had to manually untangle deeply nested Spring Boot service logic all over the place at like 2 AM cause the made up deadlines were kicking my butt.
In contrast to others, I don't need to move the goalposts - the productivity for me is here and now. Any future models will just make it better, unless we experience model collapse.
Disclaimer: you do need a LOT of code tests and validations, otherwise it all goes to shit. Maybe I'm just extending how much time it will be until it goes to shit for me as well, but go figure. You also have to babysit the models more than anyone would like or should, most of my work usually has 20-60 minutes of planning before dispatching the agent.
The way it changes the game is by lowering the cost of making radical bets so we end up trying more moonshots.
Or an example of MS - their main cost like most software companies are people, especially software devs, which are to be replaced by AI so on the surface they would greatly benefit from it. But their products are centered around helping out people do stuff on the computer. Why would you need that when the AI will do it better and faster directly operating on the data or using e.g. Python?
A product still requires a lot of handholding and human thinking, at least if one does not want everyone even throwing a glance at it to immediately be repulsed by the usual AI slop tells.
> I'm an AI advocate but that question makes me feel that AI is simply "very useful" rather than being a historical game changer for humanity.
It absolutely already is a historical game changer on par with the Industrial Revolution when it comes to the amount of jobs destroyed and economies screwed up - and the impact will be even worse in 10+ years as existing seniors retire but no new seniors rise as AI has destroyed entry level career paths.
Which economies are already screwed up?
IMO it can’t ever be on par with the Industrial Revolution because AI can only really affect the information economy. Things people do with their hands/bodies have either already been automated or can’t be with current tech. If you’d asked people decades ago they might say no one will ever work in factories by 2026 because they’ll all be automated. It didn’t work out that way. I think AI will go the same way: absolutely game changing to some industries (of which software engineering will be one) but a great many will still survive with less dramatic changes.
If anything it might result in more focus on the human aspects. How many people out there earn their stripes putting together slide decks? In a world where an AI can put together the snazziest presentation you’ve ever seen in a heartbeat it’s going to matter more how you stand at the front of the room and present those slides than it does today.