upvote
Also converting the blog from something dynamic to a static site generator. I made the same switch partly for ease of maintenance, but a side benefit is it's more resilient to this horrible modern era of scrapers far outnumbering legitimate traffic.

It's painful to have your site offline because a scraper has channeled itself 17,000 layers deep through tag links (which are set to nofollow, and ignored in robots.txt, but the scraper doesn't care). And it's especially annoying when that happens on a daily basis.

Not everyone wants to put their site behind Cloudflare.

reply
sorry if i missed it, but the original post doesn't say anything about PRs... the bots only seem to be scraping the content
reply
oh you're right, I read "pointless requests" as "PRs", oops!
reply