So the fact that the Library of Babel (i.e. all possible books) contains occasional gems means that you can't object to using it on principle? That would seem to follow from your logic.

What about a filtered set "all well formed books"? Or "all well formed books that are plausible enough that they could convince a reasonable person, regardless of their accuracy"?

It's generally taken that a cup of sewage in a barrel of wine makes a barrel of sewage. Surely a reasonable person could claim that a barrel of sewage was still sewage, even if it contained several cups of wine?