This surprised me too. I suspect the robots only got a "use mutation testing" prompt and independently decided that it must mean manual mutation testing, or possibly that they ran in a sandbox where an automated mutation testing tool was not installed.

But I'm surprised Dan Luu didn't make a remark about this. Surely he must know the difference!

Would automated mutation testing actually make a big difference for agents, or would they just find new ways to game the metric?