If your server is in Europe, and you have Australian users, there is a physical limit how low the response times can go, serving bits over that distance, so only way you can make it go below that limit, is by moving where you serve the data from closer to the user.
Lots of websites have horrible performance for whatever reasons, and lots of freelancers/consultants basically default to throwing a CDN on top of those when they get approached by businesses to fix the slow website browsing, as they're not the ones who have to pay the monthly subscription, and the less work they have to do, the better $ per hour spent for them.
I agree that it's a shit solution and there is much better sustainable ways of solving these things.
If anything else, the AI machine wants near constant streams of new data, even if it already checked with you 30ms ago.
A CDN helps immensely.
Also keeps you from getting DDoS'd if you did something like run it off your home connection.
CDN is something you do once you run out of options, not something you should reach for immediately, it makes no sense in most cases of just hosting a website.
And for large services implementing a CDN properly takes days/weeks of preparation. Once you're down it's way too late.
If your problem is AI crawlers, then simple rate limits help, I've helped countless of businesses with this already. For the ones that it isn't enough, you continue adding more roadblocks. There is no "one size fits all here" and that you seemingly is under that belief, leads less credence to what you're saying, not more.