I was not really fabricating the example I gave too much. Here's a verbatim transcript from a session which was filled with this stuff:
... thought LET'S READ RecordReader.cs RIGHT NOW!! RecordReader.cs IS WHERE EVERY SINGLE TABLE, ReadPlan, DecideValueVisibility, ReadRowCounts , AND ShowValues READS ITS DATA!! Let's read RecordReader.cs (lines 1-300)!It seems rather ironic that Anthropic givens the most logical reasoning traces, though. You'd think if they're ahead they'd either hide them or make them produce garbage (like what seems to come out of Google's models).
If you want to see example traces without jailbreaking claude or gpt, see open models that display the CoT without trying to hide it. It's pretty useful sometimes, with usual caveats (it deceptively looks like English but is heavily reward hacked, and it's generally a weak proxy for model's reasoning process).
You're of course completely right. Interestingly the "summaries" have only become more diverged over time. I now believe the big 3 are barely summarizing at all any more. I deem it likely that the small models don't get the CoT at all, they just completely make up something "plausible" solely based on the user message with a little bit of previous context. Much of the time it will be the complete opposite of something I asked for, or barely related.
Anthropic has changed this up multiple times over the last 12m, I'm talking about the iteration that was around the longest this year. I think they changed it again within the last feww weeks and potentially started actually summarizing again.
Your suggestion is the right one. Everyone who uses LLMs for their job should spend some time with open models like Deepseek and see what the thinking actually looks like.
(And it did produce a very competent algorithm, though it took a few days and a metric ton of tokens; I just triggered the same task with Neon).