upvote
I also know how RL and LLM training works :)

How many people learn one particular job? If its a job which educates 10.000 people per year, when is the financial motivation big enough to create enough training data to make them obsolete and transfer the learning into an AI model?

Also we don't know yet were the journey goes, 'dreaming' models finetuning over night might help.

Or its already just enough to have the biggest RL Loop in history of humanity: everyone of us using the chatbots and giving them signals like thumbs up and thumbs down and stuff like "DUDE WTF I TOLD YOU..." you know :)

Then we also know that the frontierlabs are paying experts and people with PhDs to generate challanges and answers.

Educating just a 1000 people every year over and over is very very expensive even if LLMs need more training data. In worst case frontierlabs will just keep investing in generating traingindata manually with their earnings and it takes. longer.

Or we find a breakthrough like how LeCun is complaining all the time that LLMs are not learning properly. We are still early and progress is still happening fast.

reply
Lol, clearly you do not know how RL and LLMs actually work.

Or maybe cut back on the acid.

reply