Given the past rate of progress, why not start being concerned now? It's a bit like the economist saying that the optimal number of flights to miss is not zero. If you keep landing short on your estimations for how far this technology will go, next time you should err on the other side.
And regarding Vending-Bench 2 (https://andonlabs.com/evals/vending-bench-2) my understanding is that models do pretty well on it now.
We're not going to stop it because of the money involved and once we're dead, it won't matter anyway, might as well just enjoy life until you're done.
We're going to get AI'd to the max, whether or not we like it or not, might as well just go with it.
It sounds like something is worth worrying about if you foresee us being dead, presumably prematurely.
I don't think your LessWrong post is going to save us.
The only way it will stop is if the wealthy / powerful people feel threatened by it, properly threatened.
The same GPU compute for LLMs runs robotic training models. Now in a few hours you can train a robot model that would have taken months 5 years ago. This model gets dumped into an actual physical robot with sensors all over and the suitability of the model is measured on robot tasks and the error in real world actions is fed back into the robot world model for further training.
> There have been experiments where an AI is given control of managing something like a vending machine
You sure you're not talking about experiments ran a couple of years ago? The more modern ones are getting wild.
https://techcrunch.com/2026/07/29/claude-opus-5-became-downr...