upvote
> all models have search capacities these days

one would hope that they disable websearch and internet access (maybe all tools?) when doing generalization testing?

reply
Is search part of the model or the harness?
reply
Maybe that's a rhetorical question but just in case - the search would always be part of the harness. A model is only handling next-token prediction for a given input. That token may be something like [[web search]] to invoke a tool call but the actual call would be handled by the harness.
reply