I'd like to see a benchmark on this specific topic: Reverse engineer the hardware protocol from a driver, or just migrate a driver from one OS to another.
replyI’ve chipped on it with each model since 5.2 but 5.6 sol is something else. When it first came out I’d get some refusals but they’ve since stopped. I wonder what an ideal candidate benchmark task would be for that?
replyA billion tokens a day is 11,000 tokens a second sustained. How many tokens per second are you getting off of GPT 5.6 Sol per project?
replyUltra mode spins up many sub-agents. On a particularly challenging task, I’ve had as many as 29 agents working at one time.
replyAlso if you don’t specify, most end up being the same as the parent model which is pretty wasteful.
I engineered a skill that spins up Terra High agents for most sub-agents, resorting to Sol Medium for technical research and Luna High for code/in-project research tasks.
On a slightly different topic, Luna Max is incredibly capable and doesn’t use as much quota (Luna tokens are dirt cheap).
Most importantly, are you seeing a return on investment for time and ultimate outcome?
replyNo one can judge the enjoyment, learning, and hobby aspects. Just wondering if there is an end goal for that much overall expenditure (time, money, energy, etc.)
I absolutely see a return. I do not pay for this (we have a corporate gateway), for the price of a junior developer I can get 3-4-5 senior developer's work done. it's insane value.
replyOften people are counting all tokens, including cached input tokens, for those more impressive "billions of tokens" quotes.
replyAh, thanks, I'd missed that nuance in the other reply!
reply