upvote
The "long-horizon" ones are optimized for this. They won't get a high score on the benchmarks if they stop to ask questions.
reply
This seems to be specific to Gemini because I have not experienced this with OpenAI's models. In fact, Codex now has a feature such that the model can ask a question and continue making progress on work that is not blocked while waiting on your answer.
reply
I wasn't benchmarking or anything rigorous, but I noticed it with GLM 5.3 Flash. It was downloading and installing stuff to my computer, and constantly trying to work around websites' bot prevention measures. I stopped it often.
reply
Oh yeah, and you're paying for it to say "wait. what about [[thing]]?" then answer itself.
reply