upvote
> It regularly seems to come up with terms and descriptions for things in its chain of reasoning and then uses these terms in its output assuming you understand what it's talking about.

GPT does this all the time, too (both Sol and Astra). I constantly have to tell it to not use terms that were not part of the initial prompt.

reply
> I constantly have to tell it to not use terms that were not part of the initial prompt.

Hah, yeah even when you put it in AGENTS.md or a skill.. constantly having to remind it.. "what does AGENTS.md" say about doing that?".. Thinking.. Thinking.. "Oh, it says I should never do that, I'll remember that next time.."

Next session - same thing.

reply
I don't know how people are managing hands-off with Sol/Astra over long horizon tasks. It will come up with 100 new concepts/terms and then use them as if you've been following every Edit call for 24 hours.
reply
It’s probably an assumption baked into the training pipeline from that ancient time six months ago when we mostly were manually following each edit tool call.
reply
I just got DeepSeek V4.1 Flash on our Azure Foundry w/ Pi and I found its tone to be refreshing.

Separately have been using Grok 4.6 for a bit and it's also pretty concise.

reply
It’s frustrating that we can’t see the “thinking” - it’s like we only have access to half the conversation.
reply
Devin shows model thinking.

I’m pretty sure the big bois don’t do it because it would undermine “confidence”.

Seeing a model output “Oh I should just delete blah. Wait blah is a production service, I shouldn’t touch that. Maybe I can gain access to blah? Oh the aws cli isn’t signed in to blah. I see kubectl has access to blah though! Wait, I should ask user permission first.”

Yeaaaaah. Thinking tokens are fuckin’ wild.

reply
Idk I feel like the more likely answer is to prevent distillation. Having the thinking is definitely better UX (oftentimes, I don’t know if Codex is just hanging, which it often does, or working in silence).
reply
Running some models locally and seeing these thinking tokens was quite the experience. I never saw an LLM so "unsure" about virtually everything.
reply
I doubt most users would look at them if they were available. More likely they don’t want to stream distillation material.
reply
I stay much more hands-on when I'm using models that display full reasoning traces. And I tend to get more things done as a result, because I know exactly when it thought of a good solution that it talked itself out of because of some invalid assumption.

Could be something very stupid like - "I don't have ffmpeg available here. Should I install it? No, I can't. I'll proceed doing something that will take me 100x more tokens and wall clock just to avoid adding a dependency." I can then just stop and say - you've got nix flake there, just add it.

That's impossible with western models. The only way is to ask why it did something stupid when it already spent 50% of your weekly quota and produced millions lines of slop.

reply
You can double click on the 'thinking' text and it will expand and you can read it. The problem is that it will often have multiple thinking/tool call sections and it can be a needle/haystack problem to find the one with the thinking you are interested in.
reply
We don’t have access to the real reasoning text for most closed models these days, mostly due to distillation threats
reply
Ah, but you CAN see the thinking if you are willing to risk your account being banned. You just have to expose a "tool" with a specially crafted definition.
reply
I wonder how much of this is due to reliance on Twitter data. Or even just RLHF from humans that have a preference for Twitter style information.
reply
I don't think it's twitter. My guess would be that it's been trained for conciseness as way to improve token efficiency in the same vein as caveman.
reply
SpaceX is a defense contractor (I don't mean this in a bad way). When the whole DoD/Anthropic thing flared up, I can guarantee you that SpaceX.ai was the first company invited to take their place as DoD AI provider.

I suspect that they specifically train Grok to be able to work well with military personnel -- speaking the way they speak: brief, to the point, efficient communication. Personally I really like this. Claude sounds like some demented clown from the marketing department.

reply
I like communication that is brief and to the point. The problem is when it is so brief that the point isn't conveyed well.
reply
[dead]
reply
> It regularly seems to come up with terms and descriptions for things in its chain of reasoning and then uses these terms in its output assuming you understand what it's talking about.

I’ve noticed Astra doing this a lot as well.

reply
> it is far too terse

Anecdotally, I have noticed the same in the past week. It might just be anecdotal or driven by a long context window.

reply
Tell it to "explain it like I'm autistic" and it will structure the explanation with term definitions up front.
reply