That is true, but the eventual realization that more machines doing more coin flips in parallel does not mean "more work gets done" might.
replyLLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker.
But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".
It's kind of weird because more machines working together does mean more work gets done. Coin flips and weighted coin flips are totally different things. Any biases weights towards reality push you closer to reality when you use them.
replyNew models keep being able to use more and more agents on longer time frames. Your hypothesis doesn't look like what we're measuring.
Who is we?
replyThe people mapping AI capabilities.
replyOh cool, so that we includes me! :)
replyMaybe turn on your light when you use a ruler? Not sure what else to say.
replyI had actually been thinking more about all the non-LLM functionality that go into the harnesses. I'm not going to name names and I haven't done any rigorous testing, but my general impression is that choice of harness matters more than choice of model. In terms of basic task completion success specifically, not code aesthetics.
replyA perfect harness will not extract gold from a dumb model. It's a system that builds on each other, though we've not probed that frontier much to have a good intuition on what effects what.
reply