It's the "way to go" for them in particular because they hide their reasoning traces, and the compaction has to account for them to be effective, so they have to do it internally, one-size-fits-all. It's not a problem with open models which can have different algorithms for different purposes, and/or any competition.

Yeah, I also think decision models for pruning context isn't worthwhile. You're paying to load it into context (of your decision model), in order to save on loading the same data into context (of your generation model). The only way the economics makes sense is if you need a much smarter model for the compaction, but can't pay for it to load all of that context (because it's uncached or something). That sounds like a niche but it really doesn't exist. Cheap models are good enough, and smart models are cheap enough.

Personally, I use a harness extension that auto-compacts my context just before the cache expires. Combined with the default compact-when-full, this ends up being pretty much always what I want. It's one less thing to think about.

I use dynamic context compaction in open code plugin. Then modified it further to include eviction.

Since I run everything local, speed benefits from keeping context low ~128k. This gets me through a single agent on multiple tasks through the same repo. It seems to break around 6M but that's more probablistic.

KV cache is definitely valuable but context poisoning happens regardless of size and it's probability increases regardless of intelligence. So at some point you're going to suffer from context overload regardless.

It's because the cache comes from the prefix. `f(f(f(A,B),C),D)` is quite different from `f(f(A,B),D)`.