Yeah, if this stuff actually worked that well already, OpenAI et al. would just run AI CEOs and engineers. Why get some other company to pay you at all when you can automate every other company out of existence and take all the money they make?
The fact of the matter is that while the tech has some uses, it sure as hell isn't a full scale replacement and you almost always actually have to massage the input into LLMs to get anything decent back out in practice. Some CEOs and managers can learn to do this, of course, and some already are... but that quickly turns into a second full time job. A "programmer" is still needed. The job might change from mostly hand-writing C++/JS/Python to prompt engineering + some manual coding to fix all the stupid fuck-ups that the bots can't solve themselves, but you still need someone to actually prompt the bot.
When that changes, it won't just be engineers losing work; there will be no reason to even have a human CEO any more.
I don't think there is any shortage of great ideas at these companies, they are just extremely bloated. And I don't think its something like indecision or bad PMs, it's "we have a finite amount of time and resources so we need to be conservative but also not too conservative"
If you have AI systems that can simply build out POCs in days, backtest on real data, show reliable results and numbers, you get a suite of product options you were never able to get before. If you have coding agents that can speed up implementation, you can build more stuff and choose the things that stick.
It changes the cost/benefit calculus of the entire business. I think you are exactly right in that: PMs/leadership are by their nature orchestration machines. Other roles are as well, but I think PM's are at a particular advantage here in that it will be quite awhile I would expect before core product decisions and creativity can be delegated to an AI, but not quite awhile until virtually everything that they're blocked on (legal approvals, POCs, wire frames, etc etc etc) will become less and less of a blocker
I'll also add this: within a large organization, you often need to interact with many different codebases owned by many different teams. Agents have made it much easier to wrangle by having the ability to deploy one to scope out your web of dependencies to learn about what would be needed for feature X, and how that integration can happen.
We've been doing far more away team work simply because it makes things move faster. It's easier to convince a team to sign off/review something than it is to get them to commit to the planning and eventual work.
It genuinely is helping things move faster inside large organizations. Or at least, it is for us, particularly since we're getting organizational prioritization to actually build the scaffolding to make those agents more effective at search.
1000x yes: you have touched on what I think is the single biggest factor here, that is the humongous value of POCs. they are gnarly to build without agents, and so we used to have to get everyone on board so we didn't get screwed in performance reviews, which was monumental task because that means convincing very busy PMs who have a lot on their plate and dont want to take risks on things they don't understand, and now it's like "can we scale this out" and you have a very nicely formatted proposal and POC. It de-risks things very quickly
Say I want to build a feature in a product.
- DS has to do a deep dive (need buy in) to opportunity size and derisk with data. That DS has to work with other DS (people may have left or moved teams) to figure out how to get the right data and figure out what the difference is between 10 different tables that have overlapping but inconsistent data. - Eng has to build up an actual simple demo (need buy in) - Design has to make it not hideous (need buy in) - Legal has to review what you're doing; POCs should involve real data where possible because otherwise no one will trust it, even if its just for user analysis on existing products
This plus about 6 internal system bugs for custom tools that are flaky and who's team has long been re-orged or laid off, 8 people who won't answer you, 2 PTO's for the stakeholders, 6 weekly meetings
no one did POCs, they just had ideas and tried to get PM's to put it on the roadmap so if it fell through at least it was bought into
The problem is they get killed by some other executive who is afraid of their department looking bad by comparison.
I think this is fairly illustrative of the challenges in AI becoming as impactful as the Internet. The bottleneck is not making things. There are plenty of people who are really good at making things and can easily be 10x or 100x as productive as the average corporate worker. YCombinator was founded on that premise - small teams of founders and early employees could be orders of magnitudes more productive than the 1000s of corporate employees at their competitors.
The bottleneck is on bringing your product to market. If your innovative new product is built within a corporate environment, it'll get killed unless the executive you work under can get a promotion out of it, and you'll be denied all sorts of help with approvals, launch process, PR, marketing, branding, etc. If it's a startup, they'll try to shut you out with exclusive distribution deals, legal threats, lobbying efforts to change the legal environment, PR campaigns, FUD, etc.
The Internet was revolutionary because it let millions of people bring products to market without asking permission. Instead of having to bid for retail shelf space among dozens of entrenched competitors that all had sweetheart deals with the retailer, you could just put up a website and sell it to anyone across the globe. Instead of following hundreds of regulations that governed existing commerce, you could just launch something and sort it out later. AI doesn't really have that property - if anything, it makes things more centralized, with more gatekeepers, and so seems more likely to destroy economic value than add to it.
I would agree but it's really minimized the building. More and more time is being spent on pre-coding work.
You'll find that most internal "innovation" teams are just lip service. In most cases, the "mothership" will be incapable of reproducing true innovation -- from a statistical perspective, culture perspective (mega corps are anti-scrappy; internal politics), and motivation perspective (startups aren't 9-to-5). It's much easier to have big M&A budgets, a VC arm, and some handwavvy internal innovation group.
Every now and again, you'll get real innovations (Waymo, transistors, GUIs), but even those have a spotty track record of commercialization when created internally.