Yeah, it's actually the case. Researchers have shown that the models response doesn't always follow from the reasoning. Whether you consider that an internal language or not really depends on what you're speculating the neural network is doing. I think there was an Antropic paper on it.
replyYou're right, it's just additional text that allows it to do thinking / reasoning-like behavior. The big proprietary models hide the real output from the user and instead provide a friendly abridged version, but that's just to protect their secret sauce from distillation.
replyThe parent is off, you’re right. They may reason in any language, typically whatever the user’s language is, and you’ll see the reasoning directly with an open model like Deepseek.
replyResearch only showed that thinking might be disconnected from the final output but in my experience they are very strongly correlated in recent models
> Research only showed that thinking might be disconnected from the final output
replyIt is trivial to regularly spot obvious contradictions and inconsistencies if you read carefully. For example I've encountered traces that amounted to "I can deduce X, therefore Y, so that means Z" but then the model turns around and outputs "the answer is W because X". It's even been demonstrated that having the model output placeholder tokens or other gibberish instead of "thoughts" still improves performance. However the thinking traces can still be useful to the end user regardless.
Current models simply generate additional text that gets added to the context for the trace. However iterative models that "think" by repeatedly looping through several layers instead of outputting text have recently been demonstrated.
reply