Right but those things exist to prevent bots. Which this is.

So at this point we're talking about participating in the (very old) arms race between scrapers & content providers.

If enough people want agents, then services should (or will) provide agent-compatible APIs. The video round-trip remains stupid from a whole-system perspective.

I mean if they want to "wriggle out" of robots.txt they can just ignore it. It's entirely voluntary.