the uncached tokens are also from runs where smol finished a task below 1024 tokens (the minimum amount of tokens needed to activate caching) which is less tokens than other harnesses are using for their system prompt (!)

> GPT-5.6 and later models: Caching is available for prefixes containing at least 1,024 tokens. This is a strict minimum.

https://developers.openai.com/api/docs/guides/prompt-caching

so in this specific case the count of uncached tokens for smol makes it look worse than it actually is

that said: it does makes sense to add more tasks that are difficult enough to fill the context window to compare the harnesses for how well they deal with compaction

staying below compaction (or with compaction at fewer compactions) is not only cheaper and faster, it also helps the agent stay on track