It’s not going to be a singularity at one moment of time. It’s not going to be instant runaway self-improvement, no matter what doomers and fetishists say.
It’s going to be gradual. We’ll see glimmers of AGI, and the “G” part will be about gradual broadening of domains and deepening of capabilities.
All of the coding harnesses are already using their own tools to self-improve, and the HITL component is getting less frequent and at higher levels of abstraction.
That’s how AGI gets here: very gradually, no hard takeoff, and nobody will be able to pinpoint when exactly it happened.
So: also no single lab with a massive advantage, no government takeovers. It’ll be a lot less dramatic than the extremes believe. IMO, of course.
Because it’s a gradient, not a binary.
Imagine all the fear- and warmongering kingmakers and powerful individuals when they realize they have no power over anything or anyone.....so game over for them. They won't like it at all, at all.
Another problem, that in order to make it understand real life, it needs robots or humans wired into it (brain interfaces) in order to test certain things in the real world. And that is another level we know almost nothing about, at least on the surface.
PS: do these self improving harnesses even work?