I've implemented a similar approach – although I'm surprised not to see mention of cache prefix busting in there!
replyI did calculations on the prefix cache effect on costs of sessions where I used it and found that the removal of tokens from context had a much bigger effect on reducing costs than cache busting had on increasing them. I should re-do that and publish it.
replyYes the idea is cool but this could really hammer usage, especially just leaving it up to the agent to decide when to do it. I'm not surprised though, considering the github account is named "Vibecodelicious" and became active in December.
replyPoking around in the repo the whole implementation is an unsupervised LLM fever-dream.
It depends on what you mean by "unsupervised" - I've been strictly working through the agent, through specs the entire time, but it's been very supervised, I just leave the mistakes in-place and have it work from there.
replyHowever at this point it can completely maintain itself. When a new version of Claude or OpenCode is released, it updates itself to work on the latest version. It can also add new implementations for harnesses pretty reliably. It's actually pretty fun to watch it at this point. "Make this work on Hermes agent and message me when you're done" and an hour later or so, I can go play with it in Hermes.
This is actually brilliant. I've seen other approaches where the context is selectively pruned, but never in a recoverable way.
replyP.S. Love the "bonsai" name btw!
ive built a variation that spins the chunks out as if they had been done by a subagent, and so the summary is like what the subagent returns
replyi couldnt figure out a nice enough way to reload the conversation, and didnt want to have to worry about the cache rebuilds so its a bit languishing
I have a hunch that OpenAIs proprietary compaction endpoint actually does something similar on the backend.
replyWhen I've asked Codex agents about things that were in their context window, they've never – to my experience – been able to actually retrieve something from before compaction when using the proprietary compaction endpoint. Instead, they've had to consult their actual transcript.
replySo... at least as of a week ago or so, I don't believe so.