This quietly assumes that "representations" of reality only encode neutral objective facts (that can be objectively judged, good luck finding a judge everyone agrees on), and "representations" does not encode intentions and value judgements, ie internalized training logic.

But that leaks in, and of course it does! If training only rewards "objective truth" we might expect that sort of neat ontological cleanliness, but we all know real training goals are more complicated.

There doesn't appear to be a bright clear distinction between objective facts and subjective values/intentions in neural networks. The distinction is perhaps illusory, just another of our preconceived notion about the nature of thought...

Fascinating to contemplate, thanks