upvote
I put context size in my Claude Code status bar and ... it does get "tired" when it's at 50%-60% context. I figured this out when I'm like "now modify the testbed to run this and add a test to exercise blah blah blah" and it was just like "That would be a substantial infra lift." In my head I say "I am paying a substantial amount for you to do this!" but just type /compact and re-prompt and my "substantial infra lift" is done without complaint.

I guess everyone needs a nap after a long day of conversing and writing code. So like us!

/compact is prone to error and I wouldn't recommend it in the middle of work. But when you are switching to a related but not completely new task, it helps. ("Now write the integration tests." vs. "on foo.go line 476 that you just wrote, I think there is a deadlock with bar.go line 123 that you just added". It doesn't really need the context to write the tests, it can get that by reading the code. But for iterating on lines of code it just produced, /compact is going to throw away whatever "thought process" led to that code and it's usually not a great thing to do.)

reply
This is one of the advantages of pi. I made a /protect command that protects the message from compaction. I also protect skills automatically.

So for long running tasks I'll do

    /protect your goal is...
reply
I am curious as to how this protect works. I am currently working on a pi brain extensions and one of the commands that I will support is:

/brains inspect

that will let you inspect a session and one of the features is the ability to review a compacted message like so:

https://gitsense.com/screenshots/inspect-pi-session-compacte...

https://gitsense.com/screenshots/inspect-pi-session-compacte...

https://gitsense.com/screenshots/inspect-pi-session-compacte...

With /compact in Pi, it creates a message that you can easily review and I am curious as to how '/protect' works.

One of the features that I am working on is to make it easy for agents to retrieve the exact message/event before compaction and I am curious if /protect is a deterministic process or if it is just instructions.

reply
I'm also a fan of Pi - although, doesn't Claude code (and most other harnesses) already have "custom summarization instructions" for compaction/summarization that already allow you to do this? Or have you added on something extra?
reply
They way I read it, this is a workaround for pi, not for Claude.

Edit: removed off topic political spam

reply
Can you please keep the unsolicited ranting out of the replies? It breaks the guidelines and degrades the value of the site. Thanks!

https://news.ycombinator.com/newsguidelines.html

reply
It's part of the rationale for why one would choose Pi over Claude, even if Pi lacks the feature. I admit my tone is entirely inappropriate though.
reply
> It's part of the rationale for why one would choose Pi over Claude

It's unrelated to my direct comment and it's a generic tangent off the thread further up, which is about technical issues with Claude Code. As the guidelines say:

"On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity."

"Eschew flamebait. Avoid generic tangents."

"Please don't use Hacker News for political or ideological battle. It tramples curiosity."

Nobody wants to see you hijacking technical threads to turn them into political flamewars. It's against the guidelines, anti-intellectual, and profoundly boring.

reply
Fair point
reply
Unfortunately I’ve used Claude and GPT models for a long time in a variety of harnesses and I agree with you and I think it’s the compaction.

For some reason, codex compaction is like black magic. I’ve never felt like I can just one one continuous thread with other models, Claude I carefully curate when I compact

reply
Compacting at all is a mistake. With 1m context window there is no reason for a single task to require compaction.

Much better to spend tokens breaking the task into chunks, documenting and storing them durably, then executing each one in clean context and just /clear after.

It’s a similar concept to compaction, just planned in advance. Much much more effective, and doesn’t burn tokens and time (“wall-clock”, Claude) doing the compaction.

reply
> With 1m context window there is no reason for a single task to require compaction

Only if money is no object. Cache reads are cheap (10% of uncached input costs) but definitely not free, and cached reads dominate session costs at long context lengths. A prompt at 20k context with $0.01 in cached reads would cost $0.40 in cached reads at 800k context, that quickly adds up for long sessions.

reply
I’m not following the implication that these economics argue for jsing compaction instead of just clearing context?
reply
Agree. Another consideration is that input token costs are effectively quadratic with the number of conversation turns. The minute you get into a long conversation you can see the costs shoot up.

You can get far more gas out of even a $20 plan of you’re careful to break things up into relatively small discrete steps, clear context regularly and give the model plenty of information to work with.

My workflow for bigger features is to write out a plan document and then proceed in smaller implementation steps, reviewing as I go. If I find something odd, I ask the agent why and often that leads to discovering a new dimension to the problem, which in turn is an opportunity to adjust the approach.

reply
You just haven’t worked on tasks that are complicated enough. Occasionally it took more than 1M tokens just to come up with a plausible plan.

Personally I find using /rewind judiciously is better than using /compact. The latter essentially gives you no control of what details to discard, but the former at least has coarse-grained control.

reply
Oh my goodness. I’ve used over 1B tokens on a single feature. I’m running at about 25B tokens/month right now.

My whole point was that by planning in advance you can shard the work into manageable sections with clear beginnings and outputs with acceptance croteria, and never compact, or even use more than a few hundfed thousand tokens in context.

It’s all hierarchical. Looking at an eval feature building right now, it’s 20ish build plans, each with zero to five or so /clear moments.

But maybe that’s the key thing… I don’t iteratively prompt ad hoc software writing. I do iterate on requirements, but if those are solid enough there is no “now write this function, now write that module”.

reply
Have you ever looked at how much performance drops as context grows? The difference in intelligence between 100k and 1M is huge, like opus drops to haiku level performance, or worse. For that reason I try to keep under 200k. That feels about the upper bound for tasks requiring accuracy.
reply
Most models’ reasoning abilities drops off significantly between the 256K-1M token ranges of the context window. There’s too much stuff to “pay attention to” at once.
reply
I auto compact around 200k tokens both due to this and because the cached read cost really escalates when sessions have more tokens than that (too short and you pay a lot in per-compact re-reading of state)
reply
This is the way.
reply
One interesting harness thing I saw Cursor do is to give the model access to the entire thread.

Even if it doesn't fit in the context window, the model can search through past turns and sanity check if something doesn't seem to be going right, or be prompted to follow an early message, "when starting on a new item, review the first message for how we should approach this"

reply
Unlike other model/harness pairs, codex+gpt also passes an opaque encrypted artifact speculated to be an embedding representing the conversation back to the successor generation which is “denser” or at least higher fidelity than summarized text.
reply
Pretty much, I stopped using Claude for that reason, I shouldn't have to keep track of the context of the model all the time. And god forbid I don't, the amount of time I've lost to Claude just sending it and then lying is not worth it when 5.5 will just do stuff, compact 2-3 times in the task and still not go insane.

This whole 1 million context window is a lie after 300k Claude degrades to unusable, and compact doesnt help. I've had Codex tabs open for weeks, I have to regularly restart Claude.

reply
That's wild to read considering Codex's compaction to me feels like the model becoming demented. Super aggressive, throws away almost everything.
reply
This is not an issue with Claude, but with Claude Code.

With swival.dev you can use long sessions without ever doing any manual compaction or reset, even with Claude models.

reply
Quality degrades long before you hit compaction, compaction is the "Next gas station, 100 miles" sign but you're already in the middle of nowhere.

You don't need to go down the rabbit hole of crazy workflows, but to avoid slop:

- Break down the work into tasks

- New context. Create a plan for one task.

- New context. Implement the plan.

- New context. /code-review the implementation

- New context. Fix the review findings.

- Repeat for next task.

I do this with Fable 5, and the quality is consistently quite good. If the context goes over 50%, the quality will become crap and you end up with 4 duplicates of the same thing across the codebase. Letting a current session review its own work is like asking a student to grade their own paper.

reply
For the vast majority of models I've worked with, 150k tokens of context seems like some magic spot. If you can stay under it then the session will stay on track and the agent won't start getting forgetful and weird.

Just like with real work, it's valuable to break down big tasks into small tasks that you can knock over in a single session. When a session does start getting too big, you just need to ask the agent to reply with a comprehensive handover report and paste it into a new session.

reply
For my quirky workflows I need 100K even to prep Opus 4.8 or higher to begin good work. Not programming, but a complex synthetic architecture for implementing more attentional autonomy for machines.
reply
deleted
reply
i found that at 700k-ish context even fable becomes an idiot, maybe openai's decision to cap codex context at 400k is correct, 400k is really a sweet spot where most part of the context is reliable.
reply
Stop using compaction. Stop treating sessions as something you keep alive for weeks. Set up a proper workflow for handing off work and start handing off well before maximum context is hit. It's less efficient, slower and more costly to constantly be sitting at high context and compaction is just not good.
reply
There is no one-size-fits-all, even for programming.
reply
There isn't, but everything that they said above is absolutely true and people who try to fight it will suffer. There is lots of flexibility within those constraints, but the constraints aren't imaginary.
reply
If you want to use LLMs effectively, everything OP said is true. It has been this way since the beginning. There will be a time when long context and compaction will be effective, but now isn't it.
reply
There's a good way and a bad way to use tools. My suggestions were based on my extensive usage of the tools in question.
reply