>it uses a lot of tokens to get work done.
replyThat's irrelevant when you use $/task as the metric, which the OP does use.
It also affects the time.
replyIt felt like a rocket compared to GLM 5.2 though. Are Chinese models generally token-heavy?
replyIf I had the GPU size, hook it up to llama.cpp and setup the --reasoning-budget and reasoning-message; Most of that additional reasoning is a lot of garbage and you can redirect it to useful output.
replyThat's how I handle the Qwen27B and 35B
> Most of that additional reasoning is a lot of garbage and you can redirect it to useful output.
replyWhat do you mean by "redirect it to useful output"? Could you give an example? This sounds interesting.
It's specific to the harness. Using dynamic context pruning, the budget cuts it off after a select amount of tokens and the budget message tells the model to use subgents to finish whatever it's thinking about
replyNice. Does it use a summarization, or a hard cutoff?
replyllama.cpp uses a hard cutoff. The agent then does "something" that is specific to the agent's implementation and configuration. It might summarize and then "finish the thought" with a different model, and then resubmit the prompt to the llama.cpp API endpoint with <think>..</think> prefilled. The primary model then infers the remainder of the reply.
reply