AI ultimately has to live in this reality and face the corresponding limitations. These companies have already consumed much of the world's supply of computing power for the next several years, and they're burning vast sums of money to keep the improvements going. RSI won't learn for free, it won't extract massive cost reductions without up front expense, it can't build factories faster than humans can work out related societal matters, it can't magically pave the deserts with solar panels for power or build and run nuclear power plants and more.
Point is, the cost of progress is already approaching the limits of what even the richest countries are able to bear (without war-like mobilization), and to bypass those constraints would require a supposed ASI to construct its own parallel supplychain from scratch without having much ability to directly interfere with reality. Recursive self improvement is ultimately limited by everything else that cannot move at the speed of electricity.
Give me one datacenter, I'll keep it under control all by myself.
Now, if some dumbasses start hooking up their data centers to...I don't know, like--robot factories? That sounds like a risk to humankind.
I'm not sure what your point is. No one thought RSI would break the laws of physics.
Specifically: AI ultimately has to live in this reality and face the corresponding limitations. These companies have already consumed much of the world's supply of computing power for the next several years, and they're burning vast sums of money to keep the improvements going. RSI won't learn for free, it won't extract massive cost reductions without up front expense, it can't build factories faster than humans can work out related societal matters, it can't magically pave the deserts with solar panels for power or build and run nuclear power plants and more.
Point is, the cost of progress is already approaching the limits of what even the richest countries are able to bear (without war-like mobilization), and to bypass those constraints would require a supposed ASI to construct its own parallel supplychain from scratch without having much ability to directly interfere with reality.
You are constructing a straw man of your own making.
Plus, "we must pace the frontier" implies that the argument is that the frontier is moving too fast, but if RSI can't move faster than the rest of reality and the models needed for RSI are already nearing the limits of current human reality, RSI can't move much faster than we can improve reality.
Yes and this was very hard and required massive real-world resources. We didn't just get a sudden flash of insight by thinking real hard about how to make ourselves smarter. Yet that's always the story that underlies any claim of RSI. You can always phrase things generally enough to make any kind of AI-led improvement look like "RSI" no matter how short-term and tightly bounded, but that's just not helpful.
Given that, it seems obvious that the next generation of LLMs will arrive faster than they would have without LLM capability. And the one after that. The floor is being raised, which makes it easier to push on the frontier.
Fable has only been out for three months. Astra is even newer. The capability of these models compared to what existed even a year ago, and the effect they are having on the production of new software, is immense.
That's all you need. RSI can happen with what we have now, just by enabling the continuous shrinking of the loop of people trying new ideas and implementing them. It does not require some magical "go make yourself better" prompt against some model that is past some magical tipping point.
Marginally easier? Yes of course, same as how it's now "easier" to write any kind of code because we aren't using punch cards anymore. That still doesn't get you to any kind of unbounded "takeoff" scenario, because diminishing returns are a thing. The "loop" of people trying out new ideas can only shrink so much.
The labs have been holding their best models back for a while it seems like.