upvote
> tokens are near-free already

How long do you think that will last?

Am over 11 billion tokens this month on a subscription. In API usage its close to $5000 and there's still 13 days left in the month.

Eventually the music in musical chairs stops. Already many anecdotal reports of the always opaque limits being lowered across the board. On a gemini pro account for personal use I can use a weeks worth of Argon in about 20 minutes on agy in the last month.

ChatGPT Pro I can use the weekly limit in under 2 days, this is with significant context clearing and handoffs.

The cost to provide frontier models isn't cheap and it's entirely subsidised by a winner-takes-all mentality from investors which in all likelihood won't come about.

reply
The cost halves every year so far. Current level of llm will get cheaper
reply
Undoubtedly you are right, except I see it the complete opposite way, companies providing the best models will probably need even more from those willing to pay in the near future, and likely get it from those wanting to ride ahead of the wave.
reply
For the majority of tasks, the best models are no longer required.

Many benchmarks are already saturated today on many tedious tasks still being done by humans across the world. Frontier labs are introducing their own budget options prior to IPO, undercutting their revenue, because otherwise they risk losing seats.

Choosing what to do and how to do it is the actual frontier now and is still not apparently well supported in any model. Current so-called decision models are not decision models, they are available-fact-evaluating models.

reply
Cost or price? ANT's shareholders and creditors aren't going to wait forever for ROI.
reply
Cost. Their margins are still pretty healthy from what they've disclosed.
reply
It's great to have that option available, but it's incredibly inefficient to just throw GPU hours at a problem that can be solved deterministically.
reply
Good for anthropic margins though. The absolute last thing they want with the upcoming IPO is people conserving tokens and ruining the already dying hockey stick narrative.
reply
It really seems too lucrative for them to weight the models toward higher token usage and less-deterministic solutions for them not to do it.
reply
> Having the equivalent of a senior engineer thoroughly review each error with full context of what the application and feature is trying to achieve could perhaps uncover different insights, or cover edge cases more smoothly.

No one is debating this. The debate is that an agent performing validation work via its own chrome plugin and internal reasoning will always cost more than performing that same validation work with deterministic tools.

reply
It isn't the same work, though, so the question is whether the added cost is worth it. Yes it costs more, but the price of tokens has fallen 100x in the past two years and it will continue to drop.
reply
"non-deterministic" -> "deterministic"

right?

reply
yuppp, thank you :D
reply
As far as timelines go both property based tests and fuzzers predate this token maxing strategy.
reply
But they are old, well tested, and not AI based, hence bland.

In this age of AI, anything well understood and effective while being efficient is out. These tokens must be used, atmosphere must be heated.

It's November. Winter is coming.

reply
Lol they keep saying that but it is heckin warm out this late November :(
reply
Normally it should snow here. I walk around with a t-shirt.

Climate change and global warming is really a scam, they have made it snow with chemtrails in the 80s, it seems.

Though I miss snow and cold. They are not making seasons as they used to anymore. They shouldn't have privatized weather.

(This comment is written by a human, and contains an implicit /s)

reply
Great, I'll mail you a quarter and you give me your Anthropic API key. Deal?
reply
> you try to predict everything that can go wrong, then design deterministic tests that can catch those things and a decision tree to figure out what to do in each case

No real software engineer think that way, as things that can go wrong are near infinite. Instead you design the right thing and write tests that verify that it’s indeed the right thing and add some more that are the wrong thing, especially at the edge of definition. Then you rely on the dev to not make very obvious errors like coding something that is not in the requirements or is completely opposite.

reply