Every defensive primitive you deploy is a potential offensive primitive for an attacker. Like the perennial denial of service issues in fail2ban type tools. You want to ban bots scanning your service, but that becomes a way for attackers to ban you from your service.

An AI which can respond offensively to a perceived attack will be abused by adversaries to reflect attacks onto their target. They will find a way to spoof attacks as seeming to come from their target and you will attack an innocent target.

That sort of countermeasure system could be done without AI as well. The problem is that it's illegal. No Castle Doctrine in cybersecurity afaik.

Interesting variation on that could be AI that builds out some sort of on the fly honeypot after identifying the attacker. Basically creating the "attack" within their own premises.