Indefinitely? Probably not.
What about when a regime wants to make the science disappear?
Becase it costs money to serve them the content.
Is the answer regulate AI? Yes.
Because when you build it you aren't, presumably, polling their servers every fifteen minutes for the entire corpus. AI scrapers are currently incredibly impolite.