upvote
But that's the whole point of the singularity. Right now the models use a lot of human effort and ingenuity to improve the models, but about a year ago it was 100% human. We'll see in another year, but if this pace continues I doubt there will be more than a handful of people who can contribute more than the models.
reply
> I think some in the AI industry drank their own Kool-Aid. They believed that if they had the best model and the most compute, they could tell the model, "Make a better model." And it would, and the next one could make its replacement, and so on.

They're not there yet. Once they get there, that's literally the definition of Singularity.

But they are getting closer. Recursive Self-Improvement used to be a phrase people mocked LessWrong crowd for using and worrying about, now it's something both OpenAI and Anthropic already publicly admitted not only to pursue, but to already be benefiting from.

reply
Sure, it's happening...but, is IT happening? By that, I mean, we can see that the models are able to iterate at a pace and scale that humans can't match, and that provides gains in model performance and efficiency. But, humans are still needed in the loop, and not just because it's necessary for safety/alignment reasons. I don't think any significant discovery has been made by models on their own, and I don't know that LLMs will ever have the capacity to invent. They can synthesize from known data amazingly well, and since they know everything "known data" is extremely broad. But, the leaps, so far, have all come from humans.

So far, I don't think the models are capable of running away on their own. Of course, it would be playing with fire to not at least consider the risks of such a runaway scenario and build in safeguards against it. But, there is no model that can build a better model on its own, thus far, to the best of my knowledge (which is far more limited than the models, so maybe I should ask them).

reply
Recursive Self-Improvement isn't instant, it starts slow and accelerates.

It starts with what they already claim to be doing - increasingly relying on existing models in non-trivial work related to training, evaluating and optimizing the next, more capable generation of models. As long as the proportion of work keeps shifting towards agents doing more and more of it, and humans less and less, that's RSI at play.

It may be that it turns out LLMs lack some fundamental level of judgement and it plateaus, but frankly I find this notion absurd; LLMs already show better judgement than most people. The alternative is, at some point LLMs will show the ability to futz their way into improvement of the next generation of models even without humans in the loop - even if much less efficient at first, if generation N+1 is more capable than generation N, it'll either take off or burn out.

reply
Those people have a lot of overlap with the LessWrong crowd. They do not have RSI now and probably never will
reply
They absolutely do, unless you believe they are lying about the fact they're using current generation models extensively to develop the next generation of their models.
reply
The companies whose insane valuation is based on accomplishing thing X say they’re getting closer to accomplishing thing X?

At least they’re led by trustworthy and honest people or we’d need to take their claims with some dose of skepticism.

reply
Is it recursive self improvement if Claude Code writes your pytorch for you?
reply
If the point of that pytorch is to improve the next generation of Claude, then yes, absolutely.
reply