upvote
> Avoid generic brevity instructions

That part is confusing because it's not like they provide an example of how default GPT-5.6 output compares with GPT-5.5 both with default output and prompted for brevity. Whenever I use such prompts, it's usually because I want the model to give me the gist in a few sentences. I'd be stunned if GPT-5.6 was that concise by default. I would think that could "break" a lot of things for developers who didn't know to make prompt changes after upgrading to 5.6. What if you were expecting GPT to be as wordy as it usually is? Then suddenly your output is not wordy enough?

Smells like OpenAI trying its best to stave off financial armageddon for another few months. Then again, I'm not sure why they chose to waste so much output computation on verbal diarrhea all this time up to now.

reply
If you conceptualize this as “there is an appropriate amount of brevity for each situation” then it would be expected for a better model to use different amounts of brevity if it gets better at determining the appropriate amount.

My view is that popular models by default output wildly excessive amounts of prose for nearly every use case, so if this changes in a new model that’s a pure win.

reply
> wildly excessive amounts of prose

Not just prose. I think this is part of the reason why you see ridiculous code with insane error handling and type checking even for impossible cases.

reply
This is one reason I switched back to Claude after testing various alternatives a few months ago. Claude ended up writing much more elegant code.

Although I was surprised that I could get very Claude like results from Chinese models though by just telling it to make the code elegant.

Reminds me of the old days with art AI where you had to put "+good -bad" in the prompt because otherwise it would assume you just wanted random quality outputs, because it had been trained on random quality inputs...

reply
The models don't get better, except when a new one is released. Their performance depends solely on the model training before release and how well you curate the context you feed it. That's it. Contrary to popular belief these things are not intelligent.
reply
>The models don't get better, except when a new one is released. Their performance depends solely on the model training before release and how well you curate the context you feed it. That's it.

Not quite. The hosting side can change reasoning budgets (or re-assign what terms like "high" means), temperature and other decoding parameters, output length limits, finetune internal "hidden" prompt, latency optimizations, finetune attention algorithms, even change quantization - all still serving as the same model.

We know (or suspect) Anthropic frequently nerfs models while keeping their name and version the same.

reply
Right. They can do all those things. And none of that will make it smart or able to learn new things. The underlying model is just an llm. But judging from the downvotes, it seems AI folks get upset when someone talks honestly about their precious piles of matrix multiplication.
reply
Intelligence can operate without learning. At a minimum inference and learning don’t need to be co-concurrent.

Not disagreeing with your point, but your terminology muddies your point.

But your point doesn't acknowledge that even with inference, there is a lot of room to tune the calculations. Multiple models, quantization tradeoffs are just the most obvious examples. Every architecture can be adjusted to increase intelligence/watt or other measure, even without further training.

reply
Might bother you to use anthropomorphic terminology like smart and learning but they are capable of producing work that traditionally required human intelligence and the whole point of gpt 3 was the ability to "learn", you can give it an example of an invented brand new coding language and it can write working code in that language
reply
Context is not the same as learning. It's easy to conflate because they're tightly coupled in our brains.

The underlying structure and tuning of the LLM are entirely unchanged by context. It merely affects the attention and activation of the network. The LLM will not be able to work with this hypothetical new language unless it is in context. This does not fit the computational meaning of learning.

Smart is not a well defined term. Nor is it's general idea formally understood. Use it freely, but you won't be saying anything meaningful unless you define your usage.

reply
Yep, people always forget that early LLMs were sold as "Zero Shot Learning".
reply
Sold as learning, but that was a marketing term, not a technical one. From a technical perspective, the LLM is not learning. Only reacting based on its original training.

You might argue that the systems we've built around them are learning in a way, as they strategically condense and save artifacts from past interactions to pass into the LLMs context. But the LLM itself, which is the source of the intelligence, is not learning. It remains entirely unchanged throughout inference. This difference may seem trite, but it has significant impacts over the long term behavior.

reply
You used the word "smart" now, whereas on the comment I replied to, you said "better".

Tuning those can definitely make a model respond better or worse.

So your claim (quoting 100% as written) that "Their performance depends solely on the model training before release and how well you curate the context you feed it" is wrong. Hence the downvotes.

Doesn't matter if LLMs are to be considered intelligent or not for the claim to be wrong.

> But judging from the downvotes, it seems AI folks get upset when someone talks honestly about their precious piles of matrix multiplication.

Often yes. In this case, it's more like they get upset when someone says something factually wrong, and then defensively changes the goalposts.

reply
> Often yes. In this case, it's more like they get upset when someone says something factually wrong, and then defensively changes the goalposts.

Oh give me a break. Show me one example of 1) any knob twisting that makes the underlying model better. or 2) any example of the AI providers twisting those knobs to do anything other than degrade performance for their own bottom line or safety.

The current post says: "it would be expected for a better model to use different amounts of brevity if it gets better at determining the appropriate amount."

When no, the model cannot "get better". It doesn't determine any appropriateness of response realtime except for the weights baked into it from the beginning and whatever context it can muster. If you cram enough guidance that it doesn't decide to ignore maybe you can make it more brief. But it (the model) can do none of those things.

LLM models are literally stupid by design.

reply
Why did you drop the first half of the sentence in your quote? The qualification there is important context for the part you did quote. And why are you talking about “better” within a model, when the sentence you quoted was talking about 5.6 vs 5.5? The post you’re referring to did not suggest a single model could “get better”. You’ve made some incorrect assumptions.

Your comments are conflating multiple kinds of “smart” and “better”. You’re right that if all the inputs are exactly the same, it takes a new model to improve (ignoring non-determinism). But the knobs and context and harness change the inputs, and they do improve output, contrary to your claim. You’re failing to capture the distinction between what the model itself does and how the harness can boost the model’s performance. It is legitimately valid and fair to call improved performance “better”, no matter where it comes from.

This all gives me the feeling you might not have experience with or understand what’s happening in today’s harness development, and the degree to which it may be as important as the weights. There are in fact a lot of things you can do to improve a model’s performance on tasks & benchmarks, without changing the model weights. @coldtea mentioned a bunch, but the harness feedback loop, internal prompts, system prompts, skills, and requests for a model to try harder, and verify and validate it’s output all lead to improved performance, all without retraining.

I agree LLMs are stupid; they’re statistical token predictors. But somehow statistical token prediction is amazing and works much better than we imagined. The talking points about LLMs being stupid token predictors are fading now because they lack explanatory power for how good the models have become. The big surprise here isn’t about LLMs. It’s about language, and how much “thinking” and intelligence is contained in language. We don’t have a good grasp on where the line is between language and intelligence. LLMs have crushed the Turing Test into dust, and yet we don’t consider them intelligent. They often appear to understand what you ask thoroughly, can re-state it in different words, they can correct your misunderstandings or add nuance you didn’t see. All this because that’s what humans do and LLMs talk like humans.

reply
> Why did you drop the first half of the sentence in your quote?

Because this entire discussion is about the release of a new model, and models are fixed. Sure you can try to modify all the scaffolding around it, but the model is the model. It doesn't matter what you're trying to improve. You can only improve the peripheral aides. And the peripheral aides can't fundamentally fix the problems with llm models when they can't learn new relationships or facts.

You will always have to wait for a new model (like this one we are talking about) for improvements to the model.

reply
> Because this entire discussion is about the release of a new model

Right. The sentence you quoted was about brevity improving with a new model. It did not suggest the model itself improving.

I’m confused why you’re stuck on this tangent. And confused why you are repeating the talking points about the model being fixed. The model is fixed - that’s true, I already agreed with you. But you don’t seem to be listening to anything else.

> It doesn’t matter what you’re trying to improve.

What do you mean? If we’re trying to improve LLM output, there are multiple ways to achieve it. A new model is one of them. Changing the inputs is another.

> You will always have to wait for a new model (like this one we are talking about) for improvements to the model.

This is true! Nobody here is disagreeing with that. The part that it seems you’ve argued incorrectly is the apparent claim that output can’t get better. Output can “improve” without improving the model.

reply
deleted
reply
> If you cram enough guidance that it doesn't decide to ignore maybe you can make it more brief.

You are now anthropomorphizing the model yourself.

reply
>Oh give me a break. Show me one example of 1) any knob twisting that makes the underlying model better.

I mentioned several.

You're now once again changing goalpoasts to say you meant the underlying model, not the overall llm performance, even though you explicitly wrote: "Their performance depends solely on the model training before release and how well you curate the context you feed it".

So, the context curation was relevant (meaning you didn't constrain your claim to the underlying model), but now somehow all the additional tunables aren't relevant (because suddenly you're just talking about the model).

End of discussion.

reply
None of what you mentioned changes the model. Because it's a fixed model. The weights are constant. It does not learn. It only knows what gets repeatedly fed to it and those fixed relationships represented by the weights. You can pretend like that's not true, but unfortunately for VCs it is true.

End of discussion.

reply
"Their performance depends solely on the model training before release and how well you curate the context you feed it".

Wrong. The face-saving backtracking doesn't change that.

reply
The models do not get better until a new one is released. And we are already at diminishing returns. So sorry. Also sorry you don't know the difference between a model and a context, harness, router, or cache.
reply
No thats probably because you misread what you were replying to and your comment was out of left field. They didnt imply models get better intra-releasally at all.
reply
I can imagine an AI insulting humans in the same way:

"The underlying model is just a biological neutral network. It seems you carbonoids get upset when someone talks honestly about synapses and neuron firing."

reply
Neural plasticity is real, and something LLMs are incapable of. So sorry.
reply
True for today’s static models during inference. Not true for self-supervised learning, not true during training or fine-tuning, of course. Ignores that LLMs might start continuous training in the future - there’s no fundamental or technical constraint that prevents LLM ‘plasticity’. And ignores that accumulating context/memories/skills/etc affects performance and might count as a valid analogy to what many people loosely call ‘neural plasticity’, which is sometimes casually mistaking knowledge for network modification.
reply
This has absolutely nothing to do with the comment you replied to.
reply
>The models don't get better, except when a new one is released.

My brother in Christ this entire thread is talking about the new model that was released

reply
It was edited. Original talked about the model learning. Glad they managed to clarify. Because the models are quite literally stupid.
reply
> the models are quite literally stupid.

You’re arguing via reductionism, and failing to explain the outcomes and emergent properties of the “stupid” system. Humans are made of atoms that are quite literally stupid, so by all means, explain our intelligence and why it’s different than LLMs. (I’m not claiming LLMs are intelligent, BTW, I just don’t think your claim helps nor believe that you can fix it.)

https://en.wikipedia.org/wiki/Reductionism#Definitions

reply
It seems like the way brevity instructions have changed is mis-aligned with how most people would expect to use them or are currently using them.

Here's the example they give:

> Instead of asking for the shortest possible answer, replace brevity instructions with prioritization:

> Lead with the conclusion. Include the evidence needed to support it, any material caveat, and the next action. Omit secondary detail and repetition.

> Keep all required facts, decisions, caveats, and next steps. Trim introductions, repetition, generic reassurance, and optional background first.

Generally speaking, when I ask for a short answer, I want a short answer because I'm not really willing to read through a bunch of bullshit to get to a summary. Putting the onus back on me to assume what the model will return and write a longer prompt detailing exactly what information I want completely misses the point of why I'm asking for a short answer in the first place.

reply
> Lead with conclusion.

I would presume (perhaps falsely?) that an instruction like this would lead to the model presenting a conclusion not supported by the evidence, and potentially backtracking as it then tries to justify said conclusion.

Yes, if deliberation happens, the model should figure out what it wants to say during that phase; but if you're using auto mode, the model is not going to be doing any deliberating half the time. In those cases, the output blathering is the model's only chance for deliberation. It "thinks as it talks", per se.

Given that, I would advise a different approach: let it blather, but then get it to write you a conclusion at the end that the model can guarantee will obviate the need to read any of the blathering.

I.e. advise the model to add an "executive summary" to the end of any non-trivial-in-length response. With some wording to carefully navigate the model between "the summary is itself too long" vs "the summary acts more like clickbait, leaving out necessary detail such that it requires actually reading the blather."

Not sure exactly what that wording would look like. I imagine something like "write your postscript executive summary as if you were a senior CIA intelligence analyst summarizing ground-level reports into a daily digest for the Joint Chiefs of Staff. Take up as little of their time as possible, but ensure that any detail critical to decision-making is retained." (But that phrasing might only be useful if the model is delivering a certain type of response, and actively counter-productive otherwise. This kind of thing is delicate.)

reply
Over hours of experimentation with various LLMs, I've found virtually any system prompt can cause unintended skewing of the model's output. Even just 5 to 8 short, direct words about length, tone or formatting can cause subtle yet significant changes in model output.

Longer, more detailed or conditional prompts always introduce an additional cognitive load as it checks every token it generates against the conditions. Making instructions more absolute (like: "Never do...") can increase the duration of compliance but at the cost of creating a significant center of attentional gravity. This can cause far more output distortion as the model devotes increasing portions of its attention budget to ensure compliance with a heavyweight requirement or prohibition. Every word in a global prompt is a trade-off between attention, compliance, drift, etc.

As someone used to thinking of computers as natural deterministic rule-followers, it's weird having to carefully wordsmith and A/B test even the simplest global prompts. It feels like coaxing a hyper-literal, emotionally sensitive, spectrum-ish toddler to comply but without being so strict it gets 'upset' or spirals into hyper-focusing.

reply
True. The real trick, if you have a client-side agent framework to hand, is to prompt it once as "gently" as possible to "just solve the problem"; and then, after its response to that, automatically prompt it again, with a separate prompt, to summarize that response a certain way. That way, the second prompt isn't "in mind" during generation of the first prompt. (And ideally, you don't even present the intermediate result to the user.)

Sadly, you can't do things like this directly using ChatGPT's own "GPTs" abstraction. (For that feature to be useful, they really need some concept of server-side agents as stateful resident IO-stream-reducer actors.)

reply
Models can be so sensitive that even prompting "Number section headings" would cause it to stop using its normal bullet point formatting anywhere. But then adding some variant of "...but don't stop using bullets as you normally do when they are needed" would make it start using bullets all the time.

Trying to craft a workable prompt got so frustrating I eventually just tried a prompt of "Don't change anything about your normal text formatting, it's perfect as is" and even that skewed the output vs no prompt. For browser chat I finally just wrote a client-side CSS UserStyle that does the formatting. Now I even have sequentially numbered sections with indented alphabetic bullets! Zero cognitive load or attentional skew and it never drifts off the formatting in long sessions.

reply
[flagged]
reply
You are absolutely correct. The second suboptimal part of the prompt is this:

> Trim introductions, repetition, generic reassurance, and optional background first.

It's not possible for the model to "trim" those before they've been output, so this is akin to telling it "not think of an elephant or even take the existence of elephants into consideration while solving this problem".

reply
You may be discounting the tokens generated in the thinking trace but not included in the output to users.
reply
That would be true of non-iterative models that just emit an output from beginning to end.

No reasonable model has worked that way for years.

reply
I would be very interested in hearing about those "iterative models" you seem so convinced have existed "for years" (so, at least since 2024 / GPT-4o). Do you have any sources?

Extraordinary claims require extraordinary proof. So far the only other people I've seen teach this prompting style or talk about models "correcting their own output" were getting their information from AI-generated, hallucinated LinkedIn and TikTok posts.

If this thing exists - which is not just a LLM outputting content serially, placed inside a harness where itself (or another llm) is prompted to review and also output revisions serially - and if a single model can be prompted to output content and "iterate" or rewind it, and it's been widespread amongst "all reasonable models", surely there will be a flurry of sources you can point me to so I can learn.

reply
Well... since o1 (Sept 2024), most models generate a "hidden" thinking trace before the visible answer... surely you have seen this by now: "Wait, that's wrong...", "Ah, now I have the full picture...". The model prunes dead ends when it composes the final answer.

Demanding sources for this is odd. It's literally been a headline feature of every frontier model for two years.

I guess you are "technically correct" that no model can "un-emit" tokens... but I don't think that is what anyone was saying or an interesting point to make.

Edit: see also this recent post, which details another place where revisions can happen, upstream of the reasoning token emission:

https://www.anthropic.com/research/global-workspace

reply
You're talking about "thinking" models which are mostly regular LLMs trained to output a "<thinking></thinking>" token delimiter before the answer that will be shown to the user, and that's a clever use of the Chain of Thought idea. All of which I'm completely familiar with.

GP was saying

> That would be true of non-iterative models that just emit an output from beginning to end.

Which suggests that there are

- "iterative" models

which

- do not output "from beginning to end"

which AFAIK is science-fiction.

reply
The disagreement here seems to be a confusion between the pure model behavior and the behavior that users experience, which is the model wrapped in a harness. It’s normal and natural now to refer to the whole system as ‘the model’ informally, even though that is technically not accurate. That ship sailed a while ago.

Since the LLM is now designed to run in a harness, it’s really not even wrong any more.

reply
don't think it's that deep mate. just sloppy wording on GP's part.
reply
This was a big concern for earlier models, but with modern CoT trained models they should be able to come to the conclusion entirely in the thinking trace.
reply
Oh the number of time LLM will, for example, be giving me the list of bugs it found in code, when I ask it for a review, just to decide there’s no big half way through explaining it.
reply
Yes this is an extremely well known result for exactly the reason you guessed. It's not just abcktracking, asking an LLM to present a conclusion and then justify is also an excellent way to provoke hallucination as the model con concts "any justification that plausibly justifies the words it's already said".

This is the actual reason why openai _invented_ reasoning models, to give them time/space to work out a solution, rather than having to magic a correct solution out of thin air from token 1.

It's less important now that all models do reasoning, but it's still almost always better to make the output come out last rather than first.

reply
I wonder if it would help to ask it to write a rough draft and then reorder it?
reply
It would (and does), yes; but this takes a lot more output tokens than asking for a summary would. The summary approach is only helpful insofar as it can be cheaper than using the thinking model. (You're basically tricking the instant model into thinking, which it can do, after a fashion.)

But, unless your desired output is literally a document for others to read, at the point where you're having a model generate a full, lengthy output multiple times over with revisions, you may as well just turn off auto mode and have it always deliberate (i.e. choose the thinking model explicitly from the model selector.) Then it'll be as messy as it needs to be while deliberating, but give you exactly what you want as output.

(And if your desired output is literally a document for others to read, that you want to interactively draft and polish, then (in the case of ChatGPT specifically) you should not only be explicitly forcing the "thinking" model, but also should be asking it to activate the "canvas" feature from the start. My understanding is that revising a canvas document involves the model emitting something like editing gestures, rather than simply re-streaming the updated chunks of text. This saves a lot of output tokens on large documents.)

reply
Why would auto mode turn off thinking?
reply
The "auto" mode is (AFAICT) a per-conversation-turn router. (Presumably via a preliminary pass through a very fast tiny model that spits out an number for how challenging it thinks the next response might be to compute.)

On high-challenge turns, the auto mode routes to the "thinking" model. But on low-challenge turns, it routes to the "instant" model.

And the "instant" model, by design, has no capacity for deliberation. (If it did, it couldn't guarantee that its responses would begin streaming "instantly.")

reply
I don't expect that would be the case. This is what's called BLUF or Bottom Line Up Front: https://en.wikipedia.org/wiki/BLUF_(communication)

The model will still have read the entirety of the document before composing its response. And I believe that even in auto mode, there are thinking tokens behind the scenes.

reply
Replace 2 word instruction ('be concise') with a 38 word instruction.

Human can no longer be concise when asking for a few sentences instead of 20 paragraphs of BS they don't want to read when all they want is a summary to verify the general direction of the prompt-work before digging into the details.

such progress!

reply
I don't know how intentional it is / was, but LLMs in general just love to hear themselves talk!
reply
They do, and I want to encourage them to do so because they think through talking. What I don’t want to do is spend time reading all that.

We will probably just get reader-side affordances for this like auto-folded justification and introduction sections and so on.

Doubtless some chat interface will add this the way they’ve added reasoning folding.

reply
Thinking models think through talking, don't reveal that talking, then answer by again thinking through talking. It's kinda funny in a way.
reply
I always expose reasoning traces. How else can you seriously debug?
reply
The closed models aren't giving you the real thinking traces, though.
reply
Those traces are just summaries of the reasoning.
reply
> LLMs in general just love to hear themselves talk!

Because that’s what’s in the training set. Reticent humans don’t have blogs.

reply
Interesting idea. I think they're getting more wordy over time, personally, so I think it's more to do with the training than the raw data.
reply
Is it just a coincidence that the companies creating them charge by the token?
reply
The aligned incentive appears to be realigning in favor of the corporation.

Pray they do not realign them further.

There are times I require single word answers. I will use whatever model responds as I desire and at this point those models are just a few.

reply
The cost-per-task benchmarks align incentives toward more efficient output and those are the ones gaining steam.
reply
I think instead of "be concise" you could tell it how long the answer should be. I.e. give the answer in one paragraph. Or in 10 lines max.

At least before it would listen to instructions like this.

reply
> At least before it would listen to instructions like this.

Would it actually follow them? IME LLMs are incapable of estimating the length of their own output, the total length of the current context, etc. They just make stuff up unless they have external tools that can inspect those things for them.

reply
That was the case for early models (Llama etc), but they got much better since then. Not perfect, but good enough.

This is from Ministral 3 14B, a 2025 model without reasoning, that you can run on your PC:

> Write a Haiku involving HackerNews, and the capability of large language models like you to reply in an exact number of words or syllables.

    Silicon whispers,
    exact words in code’s embrace—
    Haiku blooms anew.
Across multiple tries it got it wrong a couple times (by ~2 syllables). But syllables are extra tricky (because of how LLMs use tokens) and the point is that for things like "summarize in 5 bullet points" you will mostly get 5 bullet points, maybe 6, but not 10 or 20, and no need for a tool that count bullet points.
reply
For sure verbal diarrhea can be a problem. I think there's a difference between a generic instructions e.g. "be brief" and contextual guidance: "I am an experienced software developer with a recent undergraduate degree in pure mathematics. Be terse, I will ask questions if I need clarification."
reply
[dead]
reply
I think it is widely known by now that instructions to alter the LLM's "tone", things like asking it to adopt a persona ("you are the world's best programmer"), and overly broad directives ("make no mistakes") always gives poor results. Just state directly what you want. If you want something very specific, add more information. "Prompt engineering" is pseudoscience.

To put it another way, you will only get the benchmarked performance if you let it talk the way it talks by default. Trying to modify this neuters the model's IQ.

reply
It's a bit more nuanced than that. Earlier models definitely benefited a lot more from prompt engineering. I remember this distinctly from building data pipelines to do things like extract data from PDFs over the last year or two - there are numerous "tricks" like negative prompting, including the right number of examples, massaging the mock data in the JSON examples so it wasn't "too realistic", and so on. I saw how this impacted recall by running evals, so it wasn't pseudoscience.

But what has happened is the models have gotten better - which OpenAI is making explicit for some cases in this release. You need that stuff less and less as they become more human and better at inferring what's required implicitly.

You still do need to be explicit, and you probably always will, but you don't need as much "engineering" of the way you're asking for things with more recent models.

reply
> could "break" a lot of things for developers who didn't know to make prompt changes after upgrading to 5.6.

How does this differ from the other changes in behavior in 5.6 that will also break things? New models always break things.

reply
It sure is suspicious that both Anthropic (adaptive thinking) and OpenAI (Avoid generic brevity instructions) both seem to be suggesting that the best way to improve outcomes is to entirely leave it to them to decide how many tokens get used.

I mean, it's true that it would be ideal of this stuff did just get figured out optimally behind the API, but there is definitely an incentive on their side to burn more tokens.

reply
Perhaps the incentive is for variable behavior. When there is low GPU demand, burn more, but reduce when there is contention.
reply
this is a dependency update.

shouldnt you have good testing for that and not deploy a version update when those tests fail?

reply
[flagged]
reply
I'm impressed. It feels like a faster Fable (probably due to the more efficient token usage). It performs roughly the same job, just with 4x less steps (gamedev).

Remains to be seen how the "shorter prompts" advice translates to homogeneity/collapse though.

reply
and 1/4 the output quality
reply
> Avoid generic brevity instructions

y'know, I don't think I will. I really, truly want one-word answers to any binary or multiple-choice question. If I want more, I will ask for it once the model has given its answer.

reply
that is a specific brevity instruction!
reply
Control warmth[1]

> GPT-5.6 does not become meaningfully better when prompted to be broadly friendlier or more empathetic. Instead of generic instructions such as “Be friendly and warm,” use concrete guidance: > Be direct and tactful. Acknowledge friction specifically when relevant. Avoid canned reassurance and unnecessary sign-offs.

Soo basically, my new 5.6 custom instructions: Be Jeeves and eliminate all friction from my life through immense processing power. Acknowledge friction specifically when relevant. Avoid canned reassurance and unnecessary sign-offs.

[1] https://developers.openai.com/api/docs/guides/latest-model#c...

reply
> can better infer the user’s underlying goal and intended level of work

This is a trap.

It's the optimistic fallacy that poisons all "consumer scale" machine learning products and what's going to effectively ruin these models as they keep chasing it in the same way that web queries were ruined, social media feeds were ruined, and media recommenders were ruined.

For the vendor, optimizing metrics across their whole user base, they always see positive technological progress as their system gets better at making assumptions and accumulating user engagement scores in aggregate. But for the individual user, most of which has some weird tail intent/interest and some of whom have many weird tail intent/interests, the experience quietly but catastrophically degrades. Output/results become more generic, more divergent with the underspecified "weird tail" intent, and more stubbornly hard to ever wrangle towards that "weird tail" altogether.

We've been watching this cycle happen for 20 years now and it's proving hard for anybody to escape because it works so well for the trillion dollar company driving it forward. But while each step might feel ergonomic and welcome to individual users, there's a frog boiling enshitification at play.

In pursuit of output quality and capability (rather than simply the vendor's user count), what we need rather than "makes better guesses" is "presses for more clarity", even where it feels kind of annoying.

Even among human professionals, one of the first hurdles of breaking out of junior tier work is gaining the confidence to press your colleagues and clients to be more specific in their thoughts and expressions despite their desire to have you do it all for them. But they're often coming to you with incomplete, muddy, and conflicting ideas for which there is no safe and correct assumption that you might just run with, and it's your expertise (i.e. relevant "intelligence") that's critical to bringing attention to that. To achieve professional progression, you need to learn to do that and to not just optimize appeasing the ambiguous client/colleague today in exchange for mutual expense tomorrow. To avoid enshitification, which is probably not possible, we need these models to be learning that too.

reply
I agree to an extent but it needs to be balanced. Receiving a half-baked, extremely verbose recap of thinking on benign details with Opus 4.8 or GPT 5.5 feels like an extraordinary loss of quality of experience compared with fable 5.

Yes it shares less, but I think the trade-off is you pay less in tokens and hopefully it's truly just not needing to say things because it truly does just better get what you're saying, think to read X markdown file or GH issue which contains the info, etc.

As long as I can still push back and get it to share its thinking on demand and I'm confident the model isn't actually basing things on poor premises, this is okay for me. I am more productive when not inundated with time-wasting check-ins.

That said, I absolutely lament the loss of the ability to access the thinking - I would happily read the "DANGER DANGER DANGER" internal gremlin thoughts fable 5 makes to verify something if they were accessed, and prefer that to a recap presented only for my benefit.

reply
Same, I think you both have great points. Idk how you can debug effectively (the model itself) without reasoning traces
reply
[dead]
reply
It's really easy to test and it's my personal go-to benchmark. I ask the model something deep and unproven, meta physical like "oh, I heard that magic mushrooms can open the mind, but does that mean some of the great ideas people had, famous people were due to that or was the idea already there?" Like, bullshit questions that nudge towards a known example (Steve Jobs in this case) that are hard to answer and then add something like "but I'm mincing my words here, you'll get what I mean". You'll get an interesting interpretation of the question back.

I use better questions than the above but will keep my questions safe so they don't end up in the model, the point is however, when the model repeats your question back to you and "gets" what you really mean, that's a good sign of intuition and also suggests you'll get a response back that hopefully matters.

reply
I want my model to help me build up its own infrastructure that instills it with the sort of constraints I want for my project, rather than have it behave generically and automatically for everything.

It should follow instructions incredibly well while inferring contradictions or gaps in logic and surfacing those to the user as suggestions for improvements and persistence.

I really hate how Claude just assumes you want to do X/Y/Z and goes off and breaks everything and you're constantly screaming at it STOP DOING THAT. Instead, it should just do the minimal things while building its own guidance along the way in a persisted memory, like, 'would you like me to do X, now, and in the future?' etc.

reply
Yeah, all the labs seems to converging into the same (post)training for all models, while in reality, different user groups have wildly different requirements and expectations from these models.

I want the same as you, and even further, I want a model that refuses to execute changes I request if they don't make sense considering the context, or if they're impossible, and avoid any sort of quick hacks and patches. But I also want a model that does the pure opposite, that I can chuck a "Do X" query at and it figures it out. Then I'm sure there are middle-zones between these two, or even more extremes too.

But the choice isn't there, we get to chose between "fast/stupid", "medium/medium" and "slow/smart", then that's it. With system prompts we get to steer it a bit, but I've needed to make my own fork of codex to surface those things to me (the user) so I can control it better, and different models respond differently to the "Stop and don't implement anything if the request doesn't make sense yadda yadda" parts, would be lovely to have those sort of "personalities" surfaced up front when making decisions about what model to use.

reply
> ...tips for using the model:

> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”

I don't follow. Isn't "the model actually cares and will do what you say" a reason to use those kinds of instructions more liberally?

reply
Click through to the link - it states that the model tends to over correct on brevity instructions by omitting required information
reply
I think they’re saying it’s irrelevant now, possibly because it’s less likely to trail off on meandering thought bubbles.
reply
That would be contrary to a plain reading of the quote
reply
Does anyone else feel each model is like watching your kids grow up. They we're bubbly and fun and weird, you needed to tell them to sit down and be quiet.

Now if you tell them too much they go mute or stop telling you important information. Oh intelligence!

reply
Yeah, don’t do that.
reply
Not at all. Anthropomorphizing them is not productive or desirable imo.
reply
Disagree. Some of what we call "anthropomorphizing" is characterizing intelligence, human or otherwise. This reminds me of the people who used to fight against saying animals had personalities, because personalities are a "human thing" and "animals aren't conscious."
reply
I've never heard of the argument you're describing. People simply don't assert that animals lack consciousness; beyond a certain level at least, their consciousness is obvious. (Sapience, for example, is another matter.)

But this is exactly why we should not anthropomorphize the models: they are very obviously not conscious, because they are not alive, any more than conventional computer programs are. And proposing otherwise leads to absurd moral arguments, while not really serving any other purpose.

If you don't like the fact that some people disagree with you about what the word "intelligence" actually means, fine. But I am not about to entertain a world in which humans face moral retribution for "enslaving" a literal inanimate tool created by humanity.

reply
Renee Descartes famously dissected a dog in front of colleagues because he was under the impression that the howls of pain were nothing more than a mechanical sound like a bellows.

Can you prove that these models aren't conscious? And, as a counterpoint, can you prove that you are conscious, rather than a philosophical zombie?

We bred horses, cows and sheep. Most of those that live today wouldn't be alive if not for human intervention. Does that give us the right to do whatever we want with them, without consideration for feeling or morality?

In this case, you can take comfort in the idea that the tokens these models produce are likely a form of excrement to the conscious entity metabolizing the information, and rather than enslaving anything, we're creating a habitat and "harvesting" the byproducts.

reply
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”

What about my favorite, "no yapping"?

reply
It might need the longer answer to think about the question, so one approach would be to ask it normally and then ask it to repeat itself shorter.
reply
> Intent understanding

This will totally make it brain damaged over a certain tasks. Sort of like the same brain damage that prompted OpenAI project managers to destroy ChatGPT.app today.

reply
It's crazy that they sort of deprecated chat in the new ChatGPT app.
reply
Can you elaborate?
reply
It had a tough time updating today. Or this evening. It just wouldn't update. It actually just freaking disappeared from my MacBook. It took some googling and downloading and multiple tries to get it back and working. Because they also combine on a MacBook Codex with ChatGPT app. I guess codex became ChatGPT app or some silliness like that.
reply
> destroy ChatGPT.app today.

... What changed, exactly?

reply
Codex.app is gone and merged into ChatGPT.app. The upgrade process was... messy... Codex's self-update just deleted the Codex.app w/o further instruction. And ChatGPT updater failed the first time while also bricking the prior installed ChatGPT.app.

Seems good/fine once you get through upgrading the app.

reply
Wow, stories like this make me happy that I block all ChatGPT/Claude/Codex/etc updates by default, and only selectively update on demand. I do this just by setting a network rule that blocks their update check. (fortunately updates are still served on a different domain than regular usage)
reply
what's the domain?
reply
> Intent understanding

Does this mean ChatGPT will stop botsplaining things to me? I get it quite a bit more per unit time from ChatGPT than claude. Maybe that will change now.

(By botsplaining I mean when the AI explains some unstated premise of the prompt itself back at me as a correction when in many cases it's the motivation for the question in the first place)

reply
Never had that happen in ChatGPT itself, I almost always use Pro mode whenever I use ChatGPT, but what you say happens a ton in codex, when I look through the session traces it seems to happen because of the automatic compaction, where some assumption the initial pass did gets passed on as a question from the user to the part after compaction, which is a bit confusing. I think it was mentioned somewhere that the compaction got a lot better, but I haven't used GPT-5.6 enough to say if it's actually better or not on that.
reply
I found changing chatgpt's persona settings helped a lot with this!
reply
Serious question: what is a short prompt?

(For that matter at what point is it "long"? And does the rest of the context matter? Should it be short too?)

reply
Why waste time say lot word when few word do trick?
reply
I'm more concerned with things like skills
reply
It creates the context of the request without including language or terms that activate additional areas of knowledge not necessary for an accurate reply.
reply
"fix this shit"
reply
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”

I used to go to a barber and if you said "cut it short", he cut it really short.

reply
I wonder if it will do any better than past versions when one begs and pleads for it to get a job done using a concise, modest amount of code (as an expert human developer might), rather than responding to all prompts by shoveling in a large amount of code.
reply
they said it will do worse on that
reply

  > Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly.
I guess this has been achieved by training on user's chat history?
reply
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.

A shorter prompt results in half as much tokens spend? I find this very hard to believe.

reply
If it's anywhere close to the same universe as smaller models in its behavior, a lot of time in "thinking" mode is spent on reiterating on any constraints given in a prompt. So the more constraints you give it, the more tokens it will spend going "Hold on, the prompt said I have to dot my i's and cross my t's. Let me go through my work to check that all the i's are dotted."
reply
So the user must be concise, but cannot ask the model to be concise... because it hurts the model...
reply
Maybe Codex has the same problem I sometimes have focusing while reading and has to reread the same sentence over and over again.
reply
> A shorter prompt results in half as much tokens spend? I find this very hard to believe.

Should be relatively easy to test. And if it's true, just first use a very cheap near-SOTA model to first rewrite the prompt to a similar but shorter prompt before sending it to GPT-5.6.

pi.dev for example can control other harnesses.

An example: the other day for example I didn't understand why Claude Code CLI (which I hadn't used in a while) wouldn't let me cut/paste anymore (turns out they apparently fixed some long-standing scrolling and blinking SNAFU, but this modified how mouse selection/paste worked under Xorg but I didn't immediately realized they changed this)... I had to copy/paste the oauth challenge/response for I was logged out (maybe because I hadn't used Claude Code CLI in a while, dunno). But my usual copy/paste wasn't working and I didn't know how to fix it at first. And because I wasn't logged in, I couldn't use Claude Code itself for this.

My prompt was something like: "Screenshot the Claude Code TUI, transform the URL into a link, open that link in a broswer to get the oauth token, copy it character by character by simulating keypresses in the Claude Code CLI".

(remember: I had no idea how to paste with the mouse not with the keyboard, no I know but I was pissed off and wanted to be logged in immediately... So: another model / harness to the rescue).

(for the curious: it decided to use xdotool and use a 50 ms wait between simulated keypresses to copy the oauth token)

This worked just fine. And I that with a cheap model.

I think that just like Linux and Git owned many proprietary software, we'll soon have fully open-source harnesses orchestrating everything and delegating the work to proprietary tools (like "ChatGPT now Codex and vice-versa" and Claude Code)... If proprietary tools are even still needed at all.

Honestly I begin to wonder if they're even needed at all: the models, sure, while waiting for the open-weight ones to beat them. But those proprietary tools trying to lock people in?

I feel like the open source harnesses are already more powerful.

reply
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.

When has this ever not been the case? I don't think this is a GPT 5.6 specialty!

reply
Information density of the prompt is the most important factor in my experience.

And interestingly, LLMs seem particularly bad at writing prompts for other LLMs for this reason (you can guide them to be more dense, just speaking by default).

Conciseness is usually a byproduct of information density though.

reply
Lexical-priming->semantic-space-constraint;specialized-lexis+=sharp distributional-signature;∴ tight concept-cluster; generic-lexis->diffuse-activation, broad candidate-set;Attention-heads key/query-match domain-tokens;"Hamiltonian"->{operator,eigenstate,quantum,energy}->register+domain locked;Net:constrained-decoding,vocab=soft-prior over output-distribution; register-matching;#taskdef=decompress->continue
reply
Information density of the interpretability of the intent from the perspective of a human (or human-like).

If the intent is not easy to understand, it's information sparse. Because it takes a lot of CPU (or brainpower) to interpret.

You can run gzip on an English sentence to make it more textually dense, but clearly it is not more information dense in this context.

reply
I’d buy a ticket to ride the philosophical “human-like” comment with you, but I think you might have made an incorrect assumption. The model did not take longer to “decompress” the prompt than it would take for any other prompt of equal token length. If you run it with thinking enabled you might be mistaking that output as some kind of necessary gunzip step, but it’s not. Disable thinking and try again.

The prompt was also “easier to understand”, purely in the sense that the response is more or less guarantee to be what I wanted it to say, which was the point behind the demonstration. I went into more detail on it in another comment around here.

reply
I say "human-like" in the sense that LLMs are fed in text data largely in the exact form (mapped to tokens) that humans read them.

Thus from first principles it's most likely that content which is more understandable to humans is also likely to be more understandable to LLMs. Of course they are still capable of interpreting very obscure structures too, but usually at the cost of cognitive performance.

I'm open to being wrong about this, and I'm sure it's being researched.

(Specifically for text representations)

To your point, at some level of intelligence an LLM will be able to infer the intent of your prompt consistently without thinking enabled, in which case interpretability to a human matters less. But for complex tasks you aren't likely to get optimal performance with prompts that are difficult for humans to understand. And yes, you'd see that with thinking enabled as it churns over thousands of tokens trying to "mentally expand" a compressed prompt.

Interesting discussion though!

reply
Chatbot expanded this into something that made sense, but I've no idea if it's what you meant. There's an irony there somewhere.
reply
It’s what I meant, which is what I meant. Hah. The prompt and the explanation were both to illustrate the importance of domain specific lexical complexity, which is not quite the same as “information density” or necessarily “conciseness” as the OP was attributing their prompting success. It’s not that they’re wrong though. Information density requires some level of jargon and removal of unnecessary filler or scaffolding words, so my example prompt was both information dense and concise as they might say, but that’s the result, not the target. That’s confusing, but it breaks into two clearer pieces:

1. Information density is subjective, lexical complexity is how you measure it. The OP is talking “weight”, I’m talking “mass and gravity.” One of them will get you the other in most situations, so for the causal physicist it doesn’t matter, but if you’re getting into tweaking the universe then your mental model and approach matters significantly. My comment right now could be seen by some as being information dense, since I’m staying roughly on topic and tossing many concepts out, but “lexical complexity” might be the most lexical complexity in the whole thing and taken word-for-word I’m sure less than 1% of it is domain specific. “The program must use parallel processing on the CPU.” That seems decently information dense, but “the” is found in nearly every block of text ever written, “program” - are we talking television? Theater?, “must” is no better than “the”, and so on. Compare it to “#include <immintrin.h>“

2. Most people don’t realize how far that goes with LLMs. The vocabulary it has is dictated by the words in the conversation. If I ask you “what time is it?” you don’t respond “shoelace” because you’d sound crazy, although you could say it if you wanted, but the model absolutely won’t say it because that word literally does not exist yet. The end result feels the same, but the difference matters and it’s why it’s suggested not to use negating instructions. For example: “Do not mention elephants.” Well that mathematically wasn’t possible until you said it. Not having the word in the list of possibilities is a lot better than hoping it adheres to the “do not mention” part. My example prompt took that same idea from the opposite direction. The model must respond, it will be grammatically complete and coherent, and as much as possible the only words it has are the ones tightly associated with making my point for me. It didn’t ramble about baking a chocolate cake because it can’t, and making that the case is the goal with prompting, not specifically density. Word density > language density; feels similar, very different.

Perhaps this comment itself is the irony you were seeking. I spent several meandering paragraphs and included analogies to drive home the point that you should focus on the words that matter most.

reply
I've been starting with "write three paragraphs about X" when I want to talk about X, as a form of "priming the pump" - getting closer to the useful point in the phase space. After all, it doesn't matter who in the conversation generates the magic words, as long as they're present. I think your approach might be better. It's certainly enlightening. Thanks.
reply
How do you make these compressed prompts like that
reply
[dead]
reply
> LLMs seem particularly bad at writing prompts for other LLMs for this reason

Claude is terrible at this! Probably for the same reason that its writing style in prose is so annoying and full of claudisms.

reply
Claude use to be leader, too. Their metaprompt was great at the time with opus 3
reply
There was a fad a while back of building insanely long prompts - tens of thousands of tokens - including having models write prompts for themselves. I always thought it was counterproductive, especially if you're going to use the prompt more than a couple of times. (That said, the e.g. Claude Code system prompt is insanely long, so if you genuinely have a lot of information to provide maybe it's beneficial. Like, shorter is better, but you don't want to be under-specified.)
reply
For Gemini 2.5 and ~GPT5.0-5.1, longer prompts with lots of explicit instructions and examples produced better conformance. Seems like heavily second guessing the models started to get counter productive around the end of last year.
reply
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”

RIP Caveman skill. Six month good. Now skill dead.

reply
Caveman speak make compression not brevity
reply
A Yoda skill, is there?
reply
> Yoda skill, there is?

ftfy

reply
Is there a Yoda skill? -> A Yoda skill, is there?

There is a Yoda skill. -> A Yoda skill, there is.

reply
Cavemen Yoda merge can.
reply
deleted
reply
deleted
reply
do we have similar guidance or page from anthropic for claude?
reply
deleted
reply
[flagged]
reply