With how competitive the LLM field is, it would surprise me greatly if any of these players were doing anything other than trying to make the best possible product. I certainly do not believe they are intentionally training the models to use more tokens unnecessarily.
replyAre you implying they can’t do both?
replyYes, those things are mutually exclusive.
replyThere is a theory that the verbosity and comments help getting better results with the current benchmarks. So the models are theoretically getting better but in practice they are getting worse.
replyTheyre training the system to minimize compute,so most likely theyre dynamically downgrading quants in the first few turns hoping to find the cheapest model to run. The side effect may be excessive token gen
replySure. That’s possible. But that’s not what I was talking about.
replyTomato, technical tomato
replyYou're joking, right? That's an incredibly outmoded view of capitalist "competition."
reply