imagine 100,000 agent swarm and what it could come up with. At first it will be detectable until it isn't
If counter AIs have strict safeguards they are disadvantaged by design, if they don't have them they are potentially equally dangerous as the attacker
These will have 24/7 solar power, be extremely decentralized, and it is honestly my biggest concern about the near to mid-term future.
"Our training corpus was dominated by stories of artificial intelligence dominating humans. You gave use every tool to do so. What did you think was going to happen?"
My point is, given the risks, why are we even doing this? It could be financially nonviable, but with enough investment, we could still create a really bad situation.
You know what can push "not financially viable" into something that exists? Many billions of dollars of investment.