> Deepseek proposes RLVR as a way to get around the lack of $ they have to produce human reasoning trace data.
replyWhat was the difference between what deepseek did for R1 and what OpenAI did for o1?
openai did human crafted chain of thought dataset training. deepseek didn't have the resources so they attempted RL. doing RL correctly is hard because of the risk of model collapsing.
replyI don’t know why people think DeepSeek did reasoning models / RLVR before OpenAI, there was a gap of months.
replyo1 was first, and Anthropic were doing a bit of it; DeepSeek brought it to the masses, but did not invent it.
replyTotally, RLVR as a concept predates DeepSeek; but they proposed a version that was simple and scalable. Popularizing a specific version of a technique is exactly what I mean by iterations on a theme. It’s only 5% different from what others tried before, but that 5% difference showed a lot more potential than other versions of the same idea.
replySince DeepSeeks GRPO, they’ve been improvements as well like AliBabas GSPO that have gotten wide adoption. Again iterations