>Does the law really not distinguish between mechanical processing of data, and humans learning from it? It seems surprising to be if every person who read a textbook is copyright infringing. It also seems surprising if something like a lossy compression algorithm is enough to protect you from copyright law.

The google books and google thumbnails cases have so far upheld that even mechanical reproductions are allowed, depending on the context/usage.

To me the distinction hinges on the output being transformative enough to be considered a new work. I think that most of the time LLM output is.

Sometimes they go a bit wonky and overtrain on specific phrases which can result in verbatim copies of brief sections of coontent. Thats a bug, not a feature.