It's quite hard to quantify, but I think it's one shot nature really makes it hard to gauge it's capability
Friends have spoken of good days and bad coding days with me, and I find it odd nodding along, it's a strange new normal
At times it feels like we're just coding with one-armed bandits, trying to carefully line them up for a jackpot and just discarding and retrying if we don't hit
I think about some of the more complex systems I've built and I wonder how well we can build them like this
And over engineering, there seems to be over engineering everywhere, and yet, more fragility to our systems
It's all a little surreal
Or trying to reduce complexity, increasing readability and coherence of variable names (the opposite of code golf if you will), while staying within a certain limit of performance regression (e.g. "make this code as nice as you can while making it at most 0.5% slower").
Making the stuff millions of people have been using for decades better, in a way that also makes it better for humans when they read the code. Surely that's possible, some people are probably doing it but it doesn't go viral as much, because it's too mundane.
And of course, making new stuff is more exciting. I mean, you could hit on something with a vibe coded thing, and then know it's now worth to make a non-sloppy version, but you won't get much fame for making ffmpeg twice as fast by prompting an LLM. Though on the other hand, it's like a safe investment (if not in "fame", then in "improving the stuff we all have to use daily"), because you know ffmpeg and many many other things will still be around, whereas a vibe coded thing that wasn't special will be 100% forgotten the next day, or have just the one user forever.
I actually am training 2 trainees (Azubi in German) and 1 working student. All three are somewhat anxious about the future but also all are learning in a significantly increased pace, compared to the ones I worked with 1.5 years ago.
They don't have to wait for random senior to answer questions, so they get stuck way less often. They aren't allowed to use AI to generate code though, so not sure how that'd look like learning-wise if we/they went all-in on AI.
It’s been enormously alienating talking to laypeople about the apocalyptic atmosphere in the tech world, while for them ChatGPT is a cool piece of software but the world still hasn’t changed very much. They look at me bug eyed when I tell them how dramatically software engineering has changed in the matter of a couple years. We are in a bubble, or we’re just, as you say, in the eye of the cyclone which still has to hit the rest of the world.
This is insinuating that code was the bottleneck in the first place, or that every line of code is to build a new feature and not fixing existing bugs, or that apple didn't lay off enough engineers and reallocate resources to other departments to make up for the productivity boost.
I do think that companies with poor AI practices will eventually pay the piper in the form of technical debt or debilitating bugs. But let's not equate a productivity boost with a boost in releasing features, because there's plenty of business reasons to not release thousands of new features every year.
I agree with you on the rest of your points. Eventually the smoke will clear. What awaits to be seen is who is left standing when it does? I don't think I like the answer to that question.
I wish I could be so optimistic. Our lives are ruled by distorted, irrational, inefficient, failed markets, and the markets can remain distorted, irrational, inefficient, and failed for longer than we as individuals can remain solvent. "In the long term the market is a weighing machine", for term lengths that include the heat death of the universe.
But the simple fact is there's massive evidence that in skilled hands 10x or 100x engineers are possible. We're seeing evidence of it across major open source project as well. And definitely behind closed doors across companies.
Reality will catch-up with that too, once the other smoke clears.
Reduce your scale: "100x achieves in 1 hour what used to take 1 week."
One year of work could require levels of complexity and human judgement that can't be accelerated past a certain point.
1 week of work can be reduced to an hour and some change.
Besides 1h what used to to take 1 week is basically 40x given a workweek is 40h.
If one week of work can be reduced to an hour, then you should be able to complete a year's worth of work in 50 hours. If you break that into two 25-hour weeks (because a 40x dev earns the right to loaf?), what is that dev doing for the other 50 weeks in the year? What is making them so incredibly unproductive 50 out of 52 weeks in a year?
In other words, if you include everything that's required to create useful software then 100x turns out to be a fantasy.
This is especially bad with new, or quickly improving frameworks, like Android Compose. LLMs use completely outdated, deprecated APIs all the time, when they are not completely supervised. Or at least, I hope so that the framework causes it. Because if that’s not the case, then your products are fucked.
Also even with the best prompts it could never produce more working code in an hour than what I can produce in a day. Regardless of quality, just “working somehow”. Not even with an uninterrupted session. If that’s the case for some, then there is definitely also a developer skill issue. And so would definitely not trust anything coming out of their “supervision” of an LLM.
Each of these three sentences are in need of some evidence. I'm not actually seing any signs of software velocity notably increasing anywhere. Except perhaps in the AI-reseller sphere, but that seems mostly due to throwing huge amounts of VC money at it and a lack of quality control.
I still believe that. 250 lines of tight code that solves a specific problem in a way that others can maintain will always be better than 25k lines of code that's difficult to review and consume (and, therefore, becoming a liability).
A year of that is 1.3M code: the size of systemd, or postgres.
Can you imagine a single person writing systemd (not a POC, the current version feature complete and battle tested) from scratch in a year? If so can you point me to any such project?
I don't think you could get to 1.3M lines of production code, but people say AI agents are good at writing tests. I could imagine that if you had an unlimited budget you could set up agents to generate lots and lots of tests and comments, inflating your weekly total. Like, maybe you can set up an agent to loop against a code coverage tool trying to generate more and more tests to hit MC / DC levels of testing.
In the extreme / absurd case, if you could hit the SQLite ratio of 590x lines of test code vs real code then 25,000 lines of code per week could be 43 lines of production code and 24,958 lines of test code.
You'd be a "100x programmer" in terms of lines of code output, but that would not get you to SystemD levels of production code in a year.
I can't point you to a project taking that route, and I don't have the budget to try it, but I can imagine someone hitting 25k lines of code per week with lots of tests and comments padding the numbers.
I am not sure software written that way would be any good though.
I do think it is hype as a killer of knowledge work. It can certainly remove a lot of friction in the kind of borderline mechanical work that you'd formerly outsource to the lowest priced denominator, serve as an idea bouncer, remove friction for bug tracing, etc.
Attempts to cross the next line ("no need for architecture discussions, ai plans", "no need to read the code, ai reviews", and so on), nope.
As someone else mentioned, 100x is a couple days producing the outcomes (remember, not output) of a year. Or for a team, iOS delivering in a single year ten times as many features as its entire previous existence. It's not something that doesn't get noticed.
At my large org (+100 engineers), I'd say it's a mixed bag and the overall impact of AI rollout looks to be slightly negative productivity.
They probably won't say it publicly though.
It's not because some people are more productive with it that all of them are and it certainly doesn't mean that the company itself is more productive either as you have other things than code to take into account.