B) The other use of context is that it introduces entirely new information via RAG
B will never go away (as others pointed out). A, well that’s just something we’re all going to keep getting surprised at. We’ll barely give it any direction or context and the newer models will simply find the happy path.
The author is kind of suggesting that their context wasn’t really necessary to get the happy output, I think.
Chain of reasoning is a lot of context to guide token generation, but we simply see that newer models don’t need that context to get to the answer. I’m mostly reiterating this because there’s a hot take here, and that is this agentic stuff may be waived away by magic frontier-llm wand , all of a sudden.
I thought each new generation typically used more reasoning tokens?
It’s on by default, in a way. You can probably prompt these models with “and don’t reason about it, just give me the answer” and probably get a comparably good response without it using reasoning tokens for many things.