In big companies there is always waste, it's just not possible to be super efficient when you have tens of thousands of people. It's one thing in a steady state, low-competition business where you can refine and optimize processes so everyone knows exactly what their job is, but that is generally not the environment that software companies operate in. They need to be able innovate and stay competitive, never moreso than today.

The thing with AI is that it ranges from net-negative to easily brute forcing tedious things that we never have considered wasting human time on. We can't figure out where the leverage is unless all the subject matter experts in their various organizational niches really check their assumptions and get creative about experimenting and just trying different things that may never have crossed their mind before. Obviously over time best practices will emerge and get socialized, but with the rate that AI has been improving lately, it makes a lot of sense to just give employees carte blanche to explore. Soon enough there will be more scrutiny and optimization, but that doesn't really make sense without a better understanding of what is possible.

I assumed that they are saying that you spend $1k per day and that makes the developer as productive as some multiple of the number of people you could hire for that $1k.

I do not really agree with the below, but the logic is probably:

1) Engineering investment at companies generally pays off in multiples of what is spent on engineering time. Say you pay 10 engineers $200k / year each and the features those 10 engineers build grow yearly revenue by $10M. That’s a 4x ROI and clearly a good deal. (Of course, this only applies up to some ceiling; not every company has enough TAM to grow as big as Amazon).

2) Giving engineers near-unlimited access to token usage means they can create even more features, in a way that still produces positive ROI per token. This is the part I disagree with most. It’s complicated. You cannot just ship infinite slop and make money. It glosses over massive complexity in how software is delivered and used.

3) Therefore (so the argument goes) you should not cap tokens and should encourage engineers to use as many as possible.

Like I said, I don’t agree with this argument. But the key thing here is step 1. Engineering time is an investment to grow revenue. If you really could get positive ROI per token in revenue growth, you should buy infinite tokens until you hit the ceiling of your business.

Of course, the real world does not work like this.