1. A blank sheet of paper
2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
Definitely this. I've had plenty of useless meetings with no helpful outcomes that lasted an hour with 6 or more people in them, and a few of them will have had more than $4 spent on snacks, let alone everyone's wages.
That said, poor communication will always get you no matter what: had one day spent going back and forth with a client, they kept saying they wanted a button "wider" each time around, I ended up making it full screen width and they still said "wider", asked them to draw a picture and they finally said "wider vertically". That may or may not have been faster with an LLM, depending on if the LLM ever thought to ask the client to draw what they meant, but fundamentally would have been exactly the same problem.
A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time...
In my opinion, a good engineer is not good because he understands every single obscure technical detail, but because he can build solutions that the people using his technology actually want. The list of things they don't want on the other hand is endless.
Not quite as shocking as the original InstructGPT video, but surprising.
It can then raise, investigate and resolve those problems while also patching up and rationalising logging to improve future work.
The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
Could be either, but also the free version of ChatGPT seems rather braindead compared to the paid version, or to the free version of Claude.
One example: setting the agent ablaze on some task, keeping it going, and after a few days of working with the agent you have a feature. But it's not what you were supposed to make. The agent didn't have all the context and made something slightly off, and you never internalized the problem enough to even spot it. If you actually stopped to think a bit on how the feature would have to work to tie it in with everything else, like you would have done before, you would've spotted it.
Second example: something wasn't feasible to do before, due to not being worth the time investment. Like a feature that would need a bigger architectural change to implement. You and the agent take a crack at it, and rewrite huge parts of the code, and it seems to work. But now no one understands how, and it might have been the wrong way to solve it, you just went with it. Before, you would've sat down and discussed and thought about the architectural tradeoffs. Now one decision was just made with little thought.
Real life is full of unknown unknowns, AI risks putting you on the wrong path without you even realizing it.
It doesn't help a bit that (almost? I'd say all) AI products are very confident while they're guiding you on that unknown unknown path, so of course that you don't realize soon enough the epistemological situation said AI product has driven you into.
I guess that’s just the old “Build One To Throw Away” in a modern light, where prototyping more freely is now actually feasible. I think if you apply the good engineering ideas of the past decades to AI, the results can be really good!
Except I’d also add a lot of testing to the prototype, to bring up as many edge cases as possible.
Those aren't always true. Sometimes it takes a while to see the fatal flaw in an approach, and sometimes having a bunch of failed attempts isn't enough to make the right way obvious, and thinking is still required.
If oftentimes happens that you don't know that that's the wrong answer, or you don't realize it right away, for that matter, and hence that's I think that AI is a new (and very big and important) source of information entropy.
Just a recent personal example, yesterday I was curious about the historical figure that was present in an online photo taken in 1930s Romania. Google's Gemini was very quick to point out that that was (military dictator) Ion Antonescu [1], which it was obvious that he wasn't. I wrote down that that wasn't in fact Ion Antonescu, at which point Gemini was very quickly and very surely providing me the info that that guy indeed wasn't Ion Antonescu, but (former Foreign Affairs) minister Manoilescu. At this point I wasn't so sure myself, I don't have a visual image of Manoilescu in my head, so (ironically) I had to AI-dumb google for Manoilescu's available online images and it seemed like Gemini had been wrong a second time. I didn't bother asking it for a third time.
Just to put things into perspective, here in Romania it is illegal to mention Ion Antonescu's name (and propaganda image) in a positive matter, as in one risks prison for doing that, now, just imagine that the political powers that be here decide to use an AI-tool like Gemini in order to identify potential law-breakers in that respect. People would risk going to jail just because Google's Gemini was completely wrong in identifying a WW2 military dictator. Absolutely atrocious product(s).