My view is that popular models by default output wildly excessive amounts of prose for nearly every use case, so if this changes in a new model that’s a pure win.
Not just prose. I think this is part of the reason why you see ridiculous code with insane error handling and type checking even for impossible cases.
Although I was surprised that I could get very Claude like results from Chinese models though by just telling it to make the code elegant.
Reminds me of the old days with art AI where you had to put "+good -bad" in the prompt because otherwise it would assume you just wanted random quality outputs, because it had been trained on random quality inputs...
Not quite. The hosting side can change reasoning budgets (or re-assign what terms like "high" means), temperature and other decoding parameters, output length limits, finetune internal "hidden" prompt, latency optimizations, finetune attention algorithms, even change quantization - all still serving as the same model.
We know (or suspect) Anthropic frequently nerfs models while keeping their name and version the same.
Not disagreeing with your point, but your terminology muddies your point.
But your point doesn't acknowledge that even with inference, there is a lot of room to tune the calculations. Multiple models, quantization tradeoffs are just the most obvious examples. Every architecture can be adjusted to increase intelligence/watt or other measure, even without further training.
The underlying structure and tuning of the LLM are entirely unchanged by context. It merely affects the attention and activation of the network. The LLM will not be able to work with this hypothetical new language unless it is in context. This does not fit the computational meaning of learning.
Smart is not a well defined term. Nor is it's general idea formally understood. Use it freely, but you won't be saying anything meaningful unless you define your usage.
You might argue that the systems we've built around them are learning in a way, as they strategically condense and save artifacts from past interactions to pass into the LLMs context. But the LLM itself, which is the source of the intelligence, is not learning. It remains entirely unchanged throughout inference. This difference may seem trite, but it has significant impacts over the long term behavior.
Tuning those can definitely make a model respond better or worse.
So your claim (quoting 100% as written) that "Their performance depends solely on the model training before release and how well you curate the context you feed it" is wrong. Hence the downvotes.
Doesn't matter if LLMs are to be considered intelligent or not for the claim to be wrong.
> But judging from the downvotes, it seems AI folks get upset when someone talks honestly about their precious piles of matrix multiplication.
Often yes. In this case, it's more like they get upset when someone says something factually wrong, and then defensively changes the goalposts.
Oh give me a break. Show me one example of 1) any knob twisting that makes the underlying model better. or 2) any example of the AI providers twisting those knobs to do anything other than degrade performance for their own bottom line or safety.
The current post says: "it would be expected for a better model to use different amounts of brevity if it gets better at determining the appropriate amount."
When no, the model cannot "get better". It doesn't determine any appropriateness of response realtime except for the weights baked into it from the beginning and whatever context it can muster. If you cram enough guidance that it doesn't decide to ignore maybe you can make it more brief. But it (the model) can do none of those things.
LLM models are literally stupid by design.
Your comments are conflating multiple kinds of “smart” and “better”. You’re right that if all the inputs are exactly the same, it takes a new model to improve (ignoring non-determinism). But the knobs and context and harness change the inputs, and they do improve output, contrary to your claim. You’re failing to capture the distinction between what the model itself does and how the harness can boost the model’s performance. It is legitimately valid and fair to call improved performance “better”, no matter where it comes from.
This all gives me the feeling you might not have experience with or understand what’s happening in today’s harness development, and the degree to which it may be as important as the weights. There are in fact a lot of things you can do to improve a model’s performance on tasks & benchmarks, without changing the model weights. @coldtea mentioned a bunch, but the harness feedback loop, internal prompts, system prompts, skills, and requests for a model to try harder, and verify and validate it’s output all lead to improved performance, all without retraining.
I agree LLMs are stupid; they’re statistical token predictors. But somehow statistical token prediction is amazing and works much better than we imagined. The talking points about LLMs being stupid token predictors are fading now because they lack explanatory power for how good the models have become. The big surprise here isn’t about LLMs. It’s about language, and how much “thinking” and intelligence is contained in language. We don’t have a good grasp on where the line is between language and intelligence. LLMs have crushed the Turing Test into dust, and yet we don’t consider them intelligent. They often appear to understand what you ask thoroughly, can re-state it in different words, they can correct your misunderstandings or add nuance you didn’t see. All this because that’s what humans do and LLMs talk like humans.
Because this entire discussion is about the release of a new model, and models are fixed. Sure you can try to modify all the scaffolding around it, but the model is the model. It doesn't matter what you're trying to improve. You can only improve the peripheral aides. And the peripheral aides can't fundamentally fix the problems with llm models when they can't learn new relationships or facts.
You will always have to wait for a new model (like this one we are talking about) for improvements to the model.
Right. The sentence you quoted was about brevity improving with a new model. It did not suggest the model itself improving.
I’m confused why you’re stuck on this tangent. And confused why you are repeating the talking points about the model being fixed. The model is fixed - that’s true, I already agreed with you. But you don’t seem to be listening to anything else.
> It doesn’t matter what you’re trying to improve.
What do you mean? If we’re trying to improve LLM output, there are multiple ways to achieve it. A new model is one of them. Changing the inputs is another.
> You will always have to wait for a new model (like this one we are talking about) for improvements to the model.
This is true! Nobody here is disagreeing with that. The part that it seems you’ve argued incorrectly is the apparent claim that output can’t get better. Output can “improve” without improving the model.
You are now anthropomorphizing the model yourself.
I mentioned several.
You're now once again changing goalpoasts to say you meant the underlying model, not the overall llm performance, even though you explicitly wrote: "Their performance depends solely on the model training before release and how well you curate the context you feed it".
So, the context curation was relevant (meaning you didn't constrain your claim to the underlying model), but now somehow all the additional tunables aren't relevant (because suddenly you're just talking about the model).
End of discussion.
End of discussion.
Wrong. The face-saving backtracking doesn't change that.
"The underlying model is just a biological neutral network. It seems you carbonoids get upset when someone talks honestly about synapses and neuron firing."
My brother in Christ this entire thread is talking about the new model that was released
You’re arguing via reductionism, and failing to explain the outcomes and emergent properties of the “stupid” system. Humans are made of atoms that are quite literally stupid, so by all means, explain our intelligence and why it’s different than LLMs. (I’m not claiming LLMs are intelligent, BTW, I just don’t think your claim helps nor believe that you can fix it.)
Here's the example they give:
> Instead of asking for the shortest possible answer, replace brevity instructions with prioritization:
> Lead with the conclusion. Include the evidence needed to support it, any material caveat, and the next action. Omit secondary detail and repetition.
> Keep all required facts, decisions, caveats, and next steps. Trim introductions, repetition, generic reassurance, and optional background first.
Generally speaking, when I ask for a short answer, I want a short answer because I'm not really willing to read through a bunch of bullshit to get to a summary. Putting the onus back on me to assume what the model will return and write a longer prompt detailing exactly what information I want completely misses the point of why I'm asking for a short answer in the first place.
I would presume (perhaps falsely?) that an instruction like this would lead to the model presenting a conclusion not supported by the evidence, and potentially backtracking as it then tries to justify said conclusion.
Yes, if deliberation happens, the model should figure out what it wants to say during that phase; but if you're using auto mode, the model is not going to be doing any deliberating half the time. In those cases, the output blathering is the model's only chance for deliberation. It "thinks as it talks", per se.
Given that, I would advise a different approach: let it blather, but then get it to write you a conclusion at the end that the model can guarantee will obviate the need to read any of the blathering.
I.e. advise the model to add an "executive summary" to the end of any non-trivial-in-length response. With some wording to carefully navigate the model between "the summary is itself too long" vs "the summary acts more like clickbait, leaving out necessary detail such that it requires actually reading the blather."
Not sure exactly what that wording would look like. I imagine something like "write your postscript executive summary as if you were a senior CIA intelligence analyst summarizing ground-level reports into a daily digest for the Joint Chiefs of Staff. Take up as little of their time as possible, but ensure that any detail critical to decision-making is retained." (But that phrasing might only be useful if the model is delivering a certain type of response, and actively counter-productive otherwise. This kind of thing is delicate.)
Longer, more detailed or conditional prompts always introduce an additional cognitive load as it checks every token it generates against the conditions. Making instructions more absolute (like: "Never do...") can increase the duration of compliance but at the cost of creating a significant center of attentional gravity. This can cause far more output distortion as the model devotes increasing portions of its attention budget to ensure compliance with a heavyweight requirement or prohibition. Every word in a global prompt is a trade-off between attention, compliance, drift, etc.
As someone used to thinking of computers as natural deterministic rule-followers, it's weird having to carefully wordsmith and A/B test even the simplest global prompts. It feels like coaxing a hyper-literal, emotionally sensitive, spectrum-ish toddler to comply but without being so strict it gets 'upset' or spirals into hyper-focusing.
Sadly, you can't do things like this directly using ChatGPT's own "GPTs" abstraction. (For that feature to be useful, they really need some concept of server-side agents as stateful resident IO-stream-reducer actors.)
Trying to craft a workable prompt got so frustrating I eventually just tried a prompt of "Don't change anything about your normal text formatting, it's perfect as is" and even that skewed the output vs no prompt. For browser chat I finally just wrote a client-side CSS UserStyle that does the formatting. Now I even have sequentially numbered sections with indented alphabetic bullets! Zero cognitive load or attentional skew and it never drifts off the formatting in long sessions.
> Trim introductions, repetition, generic reassurance, and optional background first.
It's not possible for the model to "trim" those before they've been output, so this is akin to telling it "not think of an elephant or even take the existence of elephants into consideration while solving this problem".
No reasonable model has worked that way for years.
Extraordinary claims require extraordinary proof. So far the only other people I've seen teach this prompting style or talk about models "correcting their own output" were getting their information from AI-generated, hallucinated LinkedIn and TikTok posts.
If this thing exists - which is not just a LLM outputting content serially, placed inside a harness where itself (or another llm) is prompted to review and also output revisions serially - and if a single model can be prompted to output content and "iterate" or rewind it, and it's been widespread amongst "all reasonable models", surely there will be a flurry of sources you can point me to so I can learn.
Demanding sources for this is odd. It's literally been a headline feature of every frontier model for two years.
I guess you are "technically correct" that no model can "un-emit" tokens... but I don't think that is what anyone was saying or an interesting point to make.
Edit: see also this recent post, which details another place where revisions can happen, upstream of the reasoning token emission:
GP was saying
> That would be true of non-iterative models that just emit an output from beginning to end.
Which suggests that there are
- "iterative" models
which
- do not output "from beginning to end"
which AFAIK is science-fiction.
Since the LLM is now designed to run in a harness, it’s really not even wrong any more.
This is the actual reason why openai _invented_ reasoning models, to give them time/space to work out a solution, rather than having to magic a correct solution out of thin air from token 1.
It's less important now that all models do reasoning, but it's still almost always better to make the output come out last rather than first.
But, unless your desired output is literally a document for others to read, at the point where you're having a model generate a full, lengthy output multiple times over with revisions, you may as well just turn off auto mode and have it always deliberate (i.e. choose the thinking model explicitly from the model selector.) Then it'll be as messy as it needs to be while deliberating, but give you exactly what you want as output.
(And if your desired output is literally a document for others to read, that you want to interactively draft and polish, then (in the case of ChatGPT specifically) you should not only be explicitly forcing the "thinking" model, but also should be asking it to activate the "canvas" feature from the start. My understanding is that revising a canvas document involves the model emitting something like editing gestures, rather than simply re-streaming the updated chunks of text. This saves a lot of output tokens on large documents.)
On high-challenge turns, the auto mode routes to the "thinking" model. But on low-challenge turns, it routes to the "instant" model.
And the "instant" model, by design, has no capacity for deliberation. (If it did, it couldn't guarantee that its responses would begin streaming "instantly.")
The model will still have read the entirety of the document before composing its response. And I believe that even in auto mode, there are thinking tokens behind the scenes.
Human can no longer be concise when asking for a few sentences instead of 20 paragraphs of BS they don't want to read when all they want is a summary to verify the general direction of the prompt-work before digging into the details.
such progress!
We will probably just get reader-side affordances for this like auto-folded justification and introduction sections and so on.
Doubtless some chat interface will add this the way they’ve added reasoning folding.
Because that’s what’s in the training set. Reticent humans don’t have blogs.
Pray they do not realign them further.
There are times I require single word answers. I will use whatever model responds as I desire and at this point those models are just a few.
At least before it would listen to instructions like this.
Would it actually follow them? IME LLMs are incapable of estimating the length of their own output, the total length of the current context, etc. They just make stuff up unless they have external tools that can inspect those things for them.
This is from Ministral 3 14B, a 2025 model without reasoning, that you can run on your PC:
> Write a Haiku involving HackerNews, and the capability of large language models like you to reply in an exact number of words or syllables.
Silicon whispers,
exact words in code’s embrace—
Haiku blooms anew.
To put it another way, you will only get the benchmarked performance if you let it talk the way it talks by default. Trying to modify this neuters the model's IQ.
But what has happened is the models have gotten better - which OpenAI is making explicit for some cases in this release. You need that stuff less and less as they become more human and better at inferring what's required implicitly.
You still do need to be explicit, and you probably always will, but you don't need as much "engineering" of the way you're asking for things with more recent models.
How does this differ from the other changes in behavior in 5.6 that will also break things? New models always break things.
I mean, it's true that it would be ideal of this stuff did just get figured out optimally behind the API, but there is definitely an incentive on their side to burn more tokens.
shouldnt you have good testing for that and not deploy a version update when those tests fail?