upvote
The frontier labs want you to believe that their models are magically gaining intelligence that generalizes to new areas with each new release. But it’s an illusion: the models are post-trained on a HUGE array of hand-curated datasets for specific tasks, with good enough coverage that it feels general.
reply
The frontier labs are using all their customers' logs to train. It's the best training set there is.
reply
Time to do all business in cockney ryming slang!
reply
How is that different from you?
reply
Humans generalize much more quickly than models? We don't require millions of curated examples to learn a concept?
reply
Because there is four billion years of evolutionary pre-training.
reply
Instead of pre-training, I'd say it's more like a better architecture, training regime, higher fidelity data, and perhaps better initial "weights".
reply
All of that stuff was still trained by evolution, though.
reply
Curious, where did you see the numbers on how much frontier labs are paying for eval data?
reply
I’ve gotten recruiting emails to be a human grader of Claude coded projects. 6 month contract at $150/hour 40hrs/week. Not insanely expansive but also not cheap.
reply
>doubt on the claims that AI will become self-improving soon

It already happened. https://github.com/karpathy/autoresearch

reply