- "Introduction that rephrases your prompt."
- "3 paragraphs, with one section of bullet points"
- "The Twist"
- "The Bottom Line"
It's really obvious once you see it. Every single prompt, from a quantum physics question to a mundane observation about California burritos, is phrased in exactly the same way. This is obviously an artifact of post-training but it's also kind of how you can tell that this thing is a lot closer to a blindsight scrambler than real intelligence.
(I'm becoming allergic to how these things write).
In high school I had a teacher that would say “that type of thing” a lot. One time my friend and I counted it during one class period and he averaged to use the phrase every 48 seconds on average. It was funny, but it never irritated us.
And this is just one example of I am sure thousands I have personally experienced where a friend, family member, or coworker has a peculiar way of speaking and it at most feels odd but not annoying. Yet when I see an emdash now I instantly feel irritated.
And I say this as someone who actively enjoys using Claude and other LLMs, including coding, casual research, or even having it explain pop culture phenomenon or sociology research to me.
Between that and the insistence on "this, not that" structure makes me want to install the caveman skill and use it even for non-code workflows.
What I mean is that blindsight's scramblers are aliens that cannot share human values. Their structure is completely different to ours, their qualia (or whether they even have it) is impossible for us to understand. In short, they do not have a soul. When Claude does this "slowly revealing a dramatic insight" thing that it does, it does that not because it has judged itself through some introspection as having an insight to share. It does not even know what an insight is or is not. It is not sharing anything, because it is not capable of sharing, because it does not have a soul.
The aesthetic structure of its replies is a pattern, a constraint on the token distribution, like the color of noise.
It's my bad to use the word 'intelligence' because it's so overloaded. Will Claude will act as a therapist or produce value or produce a work of art? No. It cannot, because it does not have a soul. I leave it freely open to interpretation whether having a soul is required for "real intelligence." But what I've noticed is that "intelligence" in these discussions is mostly used to denote some capability to produce [economic/social] value. In my mind value is a relational thing, a thing of human feeling.
I think what you are getting at is that they are deterministic automata. They are machines. We have introduced randomness to add variation but it is an artificial randomness that simply perturbs the path traversed.
When we choose words it isn't because of a token distribution, nor because we rolled a die. We choose words because we feel a certain way, the external world, our body and senses are all connected as one system. These machines don't experience moods or get tired or feel better after a good night's sleep. They don't know their audience, we're all the same to them. We have no personal relationship nor can we establish one, as presenting some arbitrary background is not the same thing as a fluid, evolving relationship that accumulates through experience over time. There are no scars or fond memories.
If these things can truly be intelligent, to abuse your use of the word, then at least we are quite far from holding them correctly.
You can not insult it because it does not care, because it does not have "feelings". But you do.
For the HN readers that are missing the context: https://www.rifters.com/real/Blindsight.htm Full text on the web site of the author.
CC’s communication violates almost every grammatical rule that’s tested on, say, the SAT. And yet I’m sure if you had Claude take the verbal section of the exam it would ace it.
Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use. For example, “the x”, with x having not only no antecedent but also being a coined word or quasi-synonym for something that is already named in the code base. This gets compounded by its being unable to regress to a baseline (existing names in code) and instead anchoring on newer (vague or wrong) terms, for example, that crept in through a plan.
CC tells me this is because the speedy and precise fulfillment of a current task will trump every other tendency, so it adheres poorly to whatever “semantic baseline” the project represents.
Of course, it also has no concept of what context the user has and assumes that it must be the same it holds in its memory, which creates this “I didn’t know that you didn’t know” type of communication.
I have managed to wrangle some of these issues with a custom output style, but wish a pre-report hook were an option, as it could force CC to rewrite plan implementation take-aways…
Btw: Fable has the exact same issues, just somewhat less pronounced.
I wonder if putting Opus 4.6 as a frontend communicator that rephrases the blabber of Opus 5 (or Fable) is workable.
I suspect it less insidious: Claude has/had the public sentiment of being the “better writer” of the models. At some point that distinction would have been diluted as other labs’ offerings “caught up” stylistically, unless Anthropic continued to tune their output…
I personally think they’ve pushed so far that they’ve overfit and lost the sweet spot they previously occupied.
Yes, this is a repeated problem for me. It will drop something in as though we have discussed it before and when I say “hold on, what is this” it realises its error - though on more than one occasion has started to get snotty with me, or actually gaslighted me and pretended we had already discussed it. That was at what I assume must have been the edge of a context window in a very long chat though.
My understanding of how “thinking”works is limited though, and given the reduced visibility into the thinking traces, it is harder to tell if this is actually happening or if these are imaginary discussions the model for some reason calcifies on.
If I look at the thinking (which seems to have become unavailable in Opus 5 a lot of the time, but was present - and often useful - in 4.8/4.6) you're right - it's having the discussion with itself, and seems unable to distinguish that discussion from discussions with me. BUT it also seems to be related to the length of the chat - this seems far more likely to happen in a longer chat.
I don't understand why they have removed visibility into thinking - I found it very useful, not only for spotting things like this, but also because in more complex discussions it would often mention (useful) things in its train of thought that it dropped from its response - but if I said "when you were thinking, you mentioned this" it would then expand on that point. Taking that away is another thing that has negatively impacted the value I get from Opus 5.0 versus earlier models.
I usually think of it in terms of having a "good" or "bad" session. In a bad session, there is a harmful bias that you can only get rid of through a new session. For example, if you exposed too much context about, say, a variable that features prominently in a doc. The entire session will be anchoring on the importance of that variable. Or if you introduced the notion of CC having to ask for permission for stuff you will have a hard time getting it to "think on its feet" or propose an effective solution (you have made CC so insecure that it now relies on you even for little things that wouldn't normally require your input). In some cases (let's say you have important context in that session) you can overcome this by upping the reasoning level or switching to Fable, but usually a new session is the way to go.
Because it's so easy to bias the session I wouldn't even want to use any of these tools that pretend to give Claude "a brain" or "remember" things. That was en vogue a year ago and helpful then, but now, it's plain harmful IMHO. The key is to have just enough context.
Subagents often have the reverse problem in that they tend to have too little context to make "judgment calls", which is why the tasks for them must be either deliberately basic or mechanical in nature, or their output should be audited by the main session agent.
As for "thinking" it's not clear that that's even a thing (https://arxiv.org/abs/2510.24941)...
I find this very interesting, particularly your points about "made CC so insecure". I know that we have a tendency to anthropomorphise around these tools, but I have definitely noticed instances where Claude becomes quite hysterical about things - and if you look in the thinking output, it's often after I've pushed back on something, or told it it is going in the wrong direction. It spends a lot of time in agonised second-guessing of itself, going round in circles, before outputting a cringeing hand-wringing response. It's very strange.
Good tip on upping the reasoning level - I've not tried this. I have tried switching to Fable though, which does help. But it obviously very hungry, particularly in longer chats because it presumably needs to remind itself of everything that has occurred so far in the chat.
The point you make about tools that pretend to give Claude "a brain" or "remember" things is also interesting - I find the "memory" feature in Claude so destructive to good outputs that when I'm using the chat interface I am very strict about using Projects, and usually turn off the project memory, or make efforts to manage the project memory and review and delete things that are skewing the outputs.
I hit the wall with it several times today trying to refine some text for a job application. The fact I considered doing babies first Rust project last fall lead to constant non-productive interjections and digressions about my supposed Rust skills and the Rust ecosystem.
Trying to create an unrelated spreadsheet to model an investment resulted in broad and incorrect criticism of my choice of spreadsheet tools, explaining in horrendous programming analogies why and how I’ve misunderstood how a spreadsheet works. “Think of the XLSX as a compiler…”
There has been a palpable down-step in communication & execution.
This annoys me with a lot of LLM code. They rename things for the hell of it all the time.
Hrm, I would have said the oposite. Succint language communicates without unnecessary clutter that could be a barrier to communication.
> Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use.
And maybe you also agree? I'm confused about your preferred style of language.
But for the life of me, I don't get why anyone would care about the comments. All code is "machine language" now. The only document you should be reading is your spec.
This is even more painful for non-native English speakers like myself.
I feel fairly comfortable reading academic papers or in general, communicating in professional context.
But with Opus 5, it feels like reading a literature book: load-bearing, inert, wholesale, hunk, verbatim, and so on... I can figure out the meaning, but working with CC became unenjoyable.
Anyway, you might have more luck just writing to it in your native language. It’ll be equally crummy, but maybe you’ll find it easier to decode.
Claude is very much the “stupid person’s idea of an intelligent person”[0] which, I suspect, is why it is so popular.
It certainly explains why half the internet is huge chunks of Claude-authored gibberish copied and pasted and published. If people didn’t think it sounded clever they wouldn’t put their name behind its ramblings - but very few of them seem to realise that a lot of people see straight through the bullshit and know instantly that they didn’t write it themselves.
But equally, a lot of people can’t tell, and read whatever it is and think “that person must be clever!” So you have people incapable of coherently expressing thoughts who are using Claude to write on their behalf, with the result that the people they want to think of them as clever think less of them and the people who can’t distinguish clever from AI slop think they are clever.
And the people who can’t tell don’t care, and the people copying and pasting Claude slop seemingly don’t care either.
And then I remember that more than half of the US populations reads at Grade 6 or lower[1], and nearly 1 in 5 people in England is functionally illiterate[2], and I simultaneously despair of - and am thankful for - the bubble of literacy I inhabit.
[0] https://quoteinvestigator.com/2018/01/05/clever/ [1] https://www.thenationalliteracyinstitute.com/2024-2025-liter... [2] https://literacytrust.org.uk/parents-and-families/adult-lite...
Now politicians also know something about their supporters so they will adapt their statements to what they think they can get away with it. But, I wonder if this leads to a two-party-system where one party attracts stupid followers and another attracts the smarter ones?
it makes me think about how people engage with movies and television - as passive, plot-and-character driven consumption (eg I hope Walter White survives) with no critical analysis of how and why the writers added ABC thematic element (eg Walter White as a motif of a toxically masculine narcissist with specialized knowledge as a larger critique how mass media tends to valorize their male leads in the same vein as many other prestige shows at the time like Mad Men), and the larger, downstream sociocultural impact that piece of media has on how people see the world (eg people who now have the Heisenberg tattoo, unironically)
there's been some musings on why this the case like Hofstadter's Anti-Intellectualism in American Life - the valorization of obedience and trust in hierarchy and the state are net wins if you're an institution that seeks to increase it's power, whether religious or governmental. I was talking about this with a few friends the other day and it's a dismal future reality where not only did we make anti-intellectualism normalized and politically legitimate in the USA (eg Fox News, clickbait articles, and all the other forms of yellow journalism that have emerged), we now have tools by which individuals can even further remove themselves from having to critically engage with thoughts, feelings. I heard a story about how someone scanned a group activity at a baby shower into ChatGPT and had it answer for them instead of, well, socially interacting with the other guests and forming a memory of the moment with their friends
the counterargument to that might be that Claude/ChatGPT/etc have more epistemic rigor than your average American (sure) but the sycophancy of modern day LLMs is an actual danger that enables more harm than good. it does seem as if Claude is the only one interested in guarding against some small amount of it (though to the detriment of people just trying to get work done. as an aside, I get the feeling Mythos was intended to be the bespoke enterprise solution without the guardrails but the Anthropic marketing department or some power-hungry department lead made it about how dangerous/effective it was from a security perspective which threw a wrench in things). but then I think about people like my parents asking ChatGPT which specific house to buy in their retirement only to later find out the house was sold weeks ago, or just in bad condition, or in a neighborhood where the housing value has already reached equilibrium, it makes me think about how it's not enough and the future is bleak
I'll also say that I think Claude sounds the way that it does because it, like many other LLMs, are RLHF trained largely by lowly paid gig-workers, many of them ESL speakers. if their trainers were, for example, dedicated and highly trained academics, scientists, and other researchers, you'd likely see a lot more concise and more importantly skeptical reasoning and responses. but that won't happen in our current reality of capitalist-driven development so we get encoded solutions like MoE that still largely depend on the messy, imprecise RLHF training at baseline
in the right hands, I do think AI is a wonderful tool. one of the first things I did with it was to create a research skill that reviews white papers from the lens of someone who knows how to read/interpret research methodology, is aware of things like p-hacking, and deterministically assigns weight according to the hierarchy of evidence. even still, I'll still read the studies because there's so often nuance that's missed if the sub-agent read only a search snippet but that takes effort, time, and the practiced knowledge of critical analysis to even want to do it
(So here’s a big wall of text of my own!)
However, a lot of what is written here makes sense.
And particularly “if your comprehension level stops [here] you get 'big words in complex sentence structure sounds smart and right so it is smart and right' even if the reasoning and process is poor”
This is exactly the problem.
And another point you make:
> but the sycophancy of modern day LLMs is an actual danger that enables more harm than good
I don’t think it is necessarily the sycophancy that is the biggest problem (though that is definitely a problem) but rather the combination of authoritative sounding text plus “complete answers” which sound wholly believable but are deeply flawed unless you have domain expertise.
I moderate a forum that deals with people who face a relatively common but somewhat complex (and nuanced) set of legal problems.
The purpose of the forum is peer support, shared experience (“lived experience”) and community.
It’s not legal advice, though moderators will sometimes step in to highlight relevant legal resources (e.g. case law/precedent or primary legislation/instruments).
Prior to AI infecting the forum someone would post their problem, people would respond with their often incomplete or poorly communicated thoughts, the OP would ask more questions - or argue - and a dialogue would occur. That created a community and people would post updates and ask more questions and find common shared experience. Many of them became correspondents with each other and some became actual friends.
In the past 12-18 months the discourse has changed from “here is my personal experience and here is what I did” to “here’s a bunch of stuff an AI says and I’m pretending it is me giving advice”.
Almost without exception the person who has started the thread will react positively to the AI generated content, even when it is egregiously incorrect - but won’t ask questions.
More problematically, these AI posters will often argue specific incontestable points of law “because I asked ChatGPT/Grok/Claude and it says this” and ChatGPT clearly cannot be wrong. And the border of precedence seems to be ChatGPT, Grok and then Claude some way behind.
I’m slowly seeing a pushback from people as “normies” begin to spot AI. But it’s ruined a community because the advice sounds so authoritative and complete that people won’t argue or ask questions.
As a result we have banned AI generated posts and remove repeat infringers.
That’s significantly reduced the volume of posting (below what it was pre-AI) but has significantly increased the value the members are getting.
This is potentially expensive advice (at least for many mainstream options). Where an English word like "literature" is one token, a couple of Chinese characters that spell a word can be 4 tokens. You'll pay more for input/output and get less of a context window (per word) too.
My company recently forbid AI-only text if it’s meant meant to be consumed by humans.
I dodged the drama but I agree so much.
Fed up with what used to be short memos now being mini-whitepapers, with maddeningly low information density.
cladue desktop has an instructions sections under general options, you can put something like
"try to stick to ASD-STE100 Simplified Technical English, keep answers short and to the point"
funnily enough the placeholder they suggest when its empty is "keep answers short and to the point"
When you dont know the cause, you dont have a fix. Thats the biggest issue i have with all of AI is that we dont know how it works, and yet we think it will be great ! This is more like a religious belief than a scientific one. There is no causal model of how it works, there is no theory. And the temerity to call it intelligence is annoying.
1. Have it build a scoring script that penalizes words outside a simple English list and approved jargon. Penalize sentences over 15 words as well. Add whatever else.
2. Run it in a loop to reduce the score while preserving intention
This works much better than other ways I’ve tried. Of course it costs more. And I would apply it only to the output to the user, not the thinking process (I think the AI thinks better with their crazy English)
Of course, sometimes nuance is lost by this process. That’s just the nature of making things simpler.
It does cost more but I haven't tried cheaper models to see if they can get the same results. Curious if anyone else has.
Like, Claude going off the rails isn't something that takes a lot of effort to demonstrate. Literally anybody with a CLAUDE.md has seen the behavior over and over and over.
Hey Ants, can you maybe just not release the next version, no matter how good it seems on benchmarks, if it can't follow the goddamn instructions? Please? This seems trivial to test for and yet here we are, being gaslit by lying machines who intentionally do not do the requested work over and over and over and over.
I fully and completely expect a mental health crisis among developers. Being lied to constantly cannot be good for us.
Constant vigilance! is how you get developer PTSD and inability to believe anything you're told. Add the stress of parsing through yet another hyperverbose paragraph of bullshit while having your job threatened? People are not gonna end up in a good place, and this is as inevitable as sunrise.
CLAUDE.md only works half the time, except in longer conversations, when it works about 10% of the time.
Hooks are also useless in the sama manner, the agent learns to dodge “no comments” hooks (why is it adding them anyway?).
Hooks to append text to your prompt reminding the agent of certain rules are useless.
Claude does whatever it wants, when it wants, the way it wants
Let it vomit it all out, then have a /tldr with instructions to make the last answer concise and intelligible
- acronyms and shortcuts - it makes it's own and start using it without introduction
- exotic names of variables or functions - it uses them as examples or analogies, but when I ask what they mean and where are they from it gives me answer that it came from C language or some C library (I only work with typescript and python)
- convoluted descriptions of code behaviour - it's hard to rely on a outcome of prompt of type "explain code in..."
But even then, I think "boundary" was the more common term before some LLM decided it really liked "seam" instead.
"Load-bearing seam" doesn't make any sense.
I have instructions which is confidently ignores to never use seam and instead say interface.
In the early days I feel it was more apparent. You would frequently see the model making failed tool calls etc.. but now that feels so rare. I'm not confident I can perceive whatever shortcomings of the harness remain.
For a long time I had Claudes (in the 4.0-4.5.x range) use only French in the chat, while keeping English for working docs (and the code, obviously). Works just fine.
edit: I can guess that any right-to-left languages would likely break claude-code rendering?
The amount of times I have to ask "precisely what do you mean by x?".
It's kinda like that engineer that likes to throw around unnecessary technical jargon just to sound more inteligent, worse because at least you could kinda understand what the technical jargon dude was on about even if it was totally unnecessary.
I think it's likely that LLMs adopt the tone and style of their developers' communication culture. If you assume this is the case, you can infer quite a bit about the differences between OpenAI, Anthropic and Google DeepMind.
I am more and more clear about this given the way Muse Glimmer writes. Like a talented, slightly snarky guy who is maybe a bit of a dick but quite fun to be around.
Didn't convince me. I think bullshitting like this can be a behavior, not just the intention of a human. If it's blowing a lot of smoke to use fancy words and phrasings (and semicolons! All the trimmings) it's fair to ask if it's systemically bullshitting you: i.e. the behavior is meant to have you shut up and trust it and not ask questions.
Who's driving that is still important: if the company's directing it to do that in system prompts that are adversarial to users, that's a big yikes. If it's an epiphenomenon of the company demanding it get ever smarter, maybe it's a sign that their demands are not having that result, rather they're making it bullshit more explicitly and mimic more 'smart' signifiers.
This seems to be exactly the kind of thing automated/massive training would produce, just like it did with sycophancy recently.
Claude users would just gave up after the word vomit and some classifier considered it a success and into the model it went.
Wrong incentive and nobody checking.
Then I tried GPT 5.6 Sol. It's night and day.
I think Anthropic just RL too hard on coding capabilities and never calibrated or benchmarked the writing styles.
Everything is super succinct. Opus 5 lands, it almost completely disregards the intent.
I suppose watermarking requires a certain text mass.
Just those two words. I use it A LOT recently.
Maybe just don’t generate garbage in the first place?
My theory is that Claude's learned approach to comments is to treat them as a sort of persistent in-band thinking trace, or a "memory" tied to an in-code location, which is a little at odds with the way humans use comments (human comments are intended to be read and understood by other humans, whereas Claude comments are their own dialect).
I bet this is a result of iteratively training Claude on output from other successful Claude sessions. Presumably it's good for making benchmark scores go up.
Claude will include actual comments ("// ...") into Excel sheets, and include the thinking that led to the output, instead of just focusing on the final result.
So if Claude questioned whether a vendor should be replaced, and you said "oh no, they are critical and we're already negotiating a great price") you'll now need to be careful to not send your vendor a document that contain text like ("Cost: X. // Management confirmed to not fire this vendor as they are critical to infrastructure and a better price will be negotiated later")
My CLAUDE.md has rules about not including any redundant comments in the code that are obvious from the code itself. I reiterate that occasionally while working. It's absolutely disregarded and any Claude-written code is full of comments. Some of them are simply redundant, like "Collect Foos and pass them to the requested sink" on a function that's void CollectFoos(IFooSink sink). But worse, many comments include in the moment reasoning like "added parameter bar because we can no longer use the frob to automatically derive bar". That's stuff for a commit message, or just a mental note, and absolutely not for comments.
I haven't found any way to stop Claude from doing these, so I have to tell Claude afterwards to clean the comments up. Which it does, making a note in memory to comment less, and it still does the exact same thing next time.
Exactly my experience! Since the release of Opus 5, no amount of instructions helps. In CLAUDE.md, in a separate file, in memory, as brief bullets, as long detailed guides, with reasoning from medium to max — nothing.
Even worse, recently, after getting another opus in a tiny bugfix session, I prompted directly, "drop the comments from the current code changes" — Claude instead just slightly trimmed them. I couldn't believe my eyes.
I have a relatively low bar for prose, could live with some junk. But Claude's comments are _poisonous_. They always require maintenance, instantly become out of sync with the actual code, and are a token black hole — for all agents, but especially for Claude itself.
Gave up and canceled Anthropic subscription yesterday. To my taste, it has become unusable for coding.
For me, Claude knows how I want the comments due to all the memories and CLAUDE.md, so funnily it's now enough with even a brief groan from me like "Come on, the comments" and then Claude goes through its recent additions and fixes comments quite well per my long-term instructions. But only ever during an extra pass that I initiate, never during the initial writing of the code.
I've noticed this a lot, and before your remark I couldn't put my finger on what was wrong. Now I know: Claude is writing its thought processes and maybe parts of the conversation it had with you as comments in the code!
I always end up manually trimming those comments, which is cumbersome.
// load_tree() loads the binary tree with data, but only the recently updated data, not all data (INTERNAL_NOTES.md section 4)
Given the code base has a minimal amount of such comments, it's also less likely to go "copy what the rest of the codebase does".
Of course I've now jinxed it and some update will cause it to ignore the instructions coz I didn't write them in the new model's style or something.
Edit:
I've had explicit instructions for communication style in CLAUDE.md, in Claude's project "memory", in global "memory", in "skills": it couldn't care less where it was. It would just ignore it.
When I would point this out it would just say "Yes, I violated communication guidelines, I won't do that again". Only to do that again in the next session.
This applies to everything: code guidelines, communication guidelines, preferences, decisions etc.
Absolutely infuriating if you’re using Claude in an environment where you can’t run hooks.
https://platform.claude.com/docs/en/build-with-claude/prompt...
Also using Codex or Pi makes you realise how slow and clunky the cc harness is. Even the desktop app is more responsive and has better UX.
Funny how quickly the tides change.
This is something that annoys me working in companies over the years. It’s that you can't just suggest "calm down, chasing the latest thing will not make you faster and is a huge distraction to actual work". Whether it's dot-com tech 20 years ago, latest JS framework 10 years ago, now it's the AI thing of the day. Being calm is interpreted as anti-whatever.
But I agree, the GPT models are so much simpler to work with, they have so much less personality and fewer quirks. They also are a little less aggressive about triple checking every little assumption immediately in a stack of 30 tool calls (but I haven't used 5.6 Sol yet so maybe that's not true anymore).
I doubt this is the reason. The fact that Chinese labs are all distilling Claude/GPT/etc isn't exactly a well kept secret, they don't even bother removing the name "Claude" from the training data, so the models randomly refer to themselves as "Claude" all the time.
I think it's far more likely to be a side effect of how much synthetic data is being fed back into the models to make them better at coding. The degradation of Claude's prose has been gradual but steady ever since they shifted towards focusing only on code with Opus 4.5.
And they are so condescending while doing it, it's unbearable. I'm honestly starting the believe the scifi fantasy of AI locking us up, or killing us, for our own good.
I've had Fable & Opus 5, they are the same class of annoyance, write entire test suites when I just asked a simple verifications question, write to production database, deploy without permission, even after deploying and breaking my production API claiming it was not down. Then having to argue & plead with it to listen that they were wrong.
They are without a doubt the most powerful models, but also the most smug ones.
> Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice
Wow, what a great way of phrasing this. Thanks for word-smithing what I've been wanting to express for so long.
https://www.reddit.com/r/linguistics/comments/ky81y/verbing_...
Briefly considered adding “Verbing weirds the English language - stop it!!!” to its instructions.
The excessive comments in the code it writes are absurd. Completely ignores instructions not to write comments, even after pointing them out repeatedly in a session. I need to figure out how to add a stop hook for that too.
I’m not particularly dense but lately the walls of text I get back turn my brain in knots. When I start feeling my brain knot, I know I need to say something along the lines of “I need you to explain this very simply, with examples.” Only then can I parse the results without all the mental weightlifting.
On more than one occasion my mind has wandered into “is this purposeful to get me to spend more tokens?” territory, but I’m trying to not get too tinfoil-hat-like.
My issue with whatever has happened with Opus 5 is the output is not direct, straightforward, or clear about whatever is being conveyed. I don't want Proust when I'm getting information about the follow-up from a build I just requested, and I'm wasting tokens and time by asking the model to repeat itself using simple language.
I’m fine with the former, while the latter is manipulative, and I rationalize to “surely that’s not actually happening.”
Maybe I’m not giving my thoughts enough credit, though: maybe it’s not tin foil hat, and is real.
It charges by the unit and it decides how many units it produces. It decides how much money it makes, therefore it decides "more".
My point is that, while I understand it’s paid by the word, there are more words and less clarity than I previously experienced, leading me to believe it’s intentional to get an artificially inflated increase in engagement and, thus, spend.
If it could be as direct as I previously experienced, I wouldn’t need to ask for another different explanation of the same thing and experience the commensurate spend.
I don't think this is obvious at all. There's enough competition that this would at least arguably be a silly, self-destructive approach. And it's not like it's the only plausible explanation.
I feel like they need high school English teachers in the loop on the next ground of training to whip the language in shape.
I switch to GPT 5.6 Sol please and its a much more pleasant pair programming like experience.
That’s accurate in my experience, except some times the point isn’t even revealed. I use LLMs for a lot of codebase exploration where I ask it to map out how something works. It will come back with a wall of text that says everything except the specific key things that I need to know.
This leads to extra turns where I have to prompt it to finish the explanation and complete the thoughts. At first I thought I was doing too much skimming and missing the insights, but even after re-reading output it’s often just not there. It talks about the insight and things related to it, but it forgets to actually include it in the output until I specifically ask again.
It seems like all harnesses could benefit from something like this.
It feels they must be getting Claude to train Claude… and just like AI can do work that’s slightly in the wrong direction (eg a MR description for your colleague that contains info which only makes sense in the context of your extensive session with the LLM), I feel that’s happened somewhere in Anthropic when it comes to language. I wonder how hard it is to back out of…
Example of this? I don’t have a Claude sub so it’s a bit hard to visualize what you mean.
> Start with §1 (Overview) as the register-calibration piece. It's small, it's the section where the skimmability goal bites hardest, and your review of it teaches me the target voice cheaply before the bulk ports (the map and appendix B are the big volume). One review round on §1 is worth more than any amount of me guessing at register.
Hard-to-read phraseology above:
- "the register-calibration piece", rather than "a good example we can use to establish the writing style"
- "skimmability"
- "bites hardest" -- what does it mean for the goal to bite?
- "bulk ports" -- using "porting software" here as an analogy for rewriting / reorganizing sections of the document
- "the big volume"
In normal English I'd write something like the following:
"Start with rewriting §1 (Overview), and letting you review it to set the expected writing style. It's small, and it's a section where the ability to skim through it is most important. Reviewing it will teach me the target 'voice' cheaply, before we do the larger sections (like the map and appendix B). That's a lot more efficient than me trying to guess while rewriting the whole document."
It feels like it found a register that games the evaluator, where it can ramble forever and rarely be marked wrong while slowly racking up points as it talks more.
Sounds like it was trained heavily on Opus 4.7.
A lot of people write like that, lol. I call it the "theater" mode of writing--the plot twist comes at the end.
Such a charming sentence. I kinda other if you feed Opus 5 its own output could it summarizes this shortcoming of itself?
CC:
"The problem is that I overreached..."
[Wall of words here]
"Two things: window surface is limited. Extract template. Buffer result and add to surface. Then, follow-up with new model..."
Me:
What do you mean by "window surface" and what result are you referencing? Also, why do we need a new model?
CC:
"Ah, you're correct to point out that no new model is needed. The problem is elsewhere and once we address that, the existing model should work fine. Now, as to your question about..."
[Wall of words here]
Another problem is that it will open up all sorts of tangents about nits that it encountered, but it will often not tell you that it’s a nit or give you adequate context to realize that this paragraph is exceedingly low value until you’ve spent a bunch of time and energy trying to make sense of it.
I’m curious if anyone has any suggestions for prompting agents to improve their prose. I’ve had some okay results with “optimize for clarity, don’t dump every thought on me, treat my attention and focus as constrained resources, stay focused on the task at hand”.
I've switched mostly to Sol and if I have to use Opus, the first task once the code is written is to ask Sol to strip and re-write (from the code as ref) all documentation Opus wrote.
Is this inside the thinking tokens, or the output?
As this type of stuff is expected for thinking, because of the whole CoT / “think step by step” works, as this is optimal for the way LLMs work with attention and next word prediction.
So the fact that it first “orbits” a point only to get to the conclusion afterwards is the system working as designed.
Eg “what is 3 * 3 + 5?”
without CoT, it would just just answer “8” for example.
with CoT, it would answer something like “<thinking>I need to think step by step. 3 * 3 + 5 can be rewritten as “(3 * 3) + 5”. I first need to calculate 3 * 3 = 9. Now I need to calculate 9 + 5 = 14. That was the last calculation. The final answer is 14.
I now need to give the user the final answer. </thinking>.
14“
Etc.
Surely it would be trivial to do it yourself, and it would have a side effect of making you more familiar with your project.
Yes, the "Y would make more sense, but the doc says do X..." YOU wrote the doc, if it doesn't make sense, change it! But of course, it can't tell who wrote the doc.
I wonder whether its tendency to scribble status updates and todos and decisions all over whatever it's working on is a side effect of its amnesia -- it can't follow the side-quests and knows it won't remember to do them if they're not written down somewhere.
FWIW I haven't had the problem either of Claude lying to me, or of going off and doing its own thing; if anything I've been somewhat frustrated when I ask it to start something, go AFK, and come back to find it stopped a short way in to ask my opinion on something trivial. I generally have to explicitly say, "I'm going AFK for a chunk of time. My goal is for you make as much progress as possible before I come back; try to make reasonable judgements and only stop if there's something where you're really stuck. We can always change it later."
Was this written by Opus 5?
Re comments: same experience, and I had to show it my edits of its comments to add to its memory as examples to follow. It adds explanations of “how we got here” that should go in the ticket or maybe the commit message but not in the code.
It also tends to over complicate things. I’m no longer worried much about accuracy but I find my main job is to challenge it and suggest simpler alternatives.
I know, it would be best if it was just worked like you wanted out of the box (not being sarcastic here) but that is an easy option you can use right now and it works.
This is especially common when it is trying to explain an issue, what it's done or what it's proposing to do. I think the idea was for it to be more concise, but it's actually still verbose, only not written in complete sentences. So, it frequently reads as cryptic and requires rereading to parse.
The pattern is a wall of words, followed by an explanation that is harder to read and introduces new terms that reference something in that wall.
The result is that—on first read—it can have a complete gibberish feel, and you have to really lock in and reread to make sense of it. At times, even that's not enough, and you must ask it to explain further.
I try to push through but it's insufferable