upvote
Or you actually do understand it.

You can't just assume smaller is better but it often is. And very often it's more information-dense.

reply
You can compress syntax, losslessly even, with zero understanding of its semantics. Zero understanding not only imbued into the compressor/decompressor, but even the designer of the compressor doesn't require understanding the semantics. Actually, even of the syntax.

A compression program can compress a book written in a language that the author of the program doesn't understand, on a topic he knows little about.

reply
Finding common characters and building a list of words is a low level type of understanding. Doing it better does actually start directly representing syntax patterns and that's a less-low level of understanding.

I think "losslessly even" is the wrong way to think about it. Lossless compression often requires less understanding than high quality lossy compression. If you can do a lossy compression that correctly decides what details are unimportant, that's a good sign of understanding.

reply
> I think "losslessly even" is the wrong way to think about it. Lossless compression often requires less understanding than high quality lossy compression. If you can do a lossy compression that correctly decides what details are unimportant, that's a good sign of understanding.

This is the crux and reminds me of things like mp3 that exploit the nature of human hearing being limited to a frequency range.

reply
We model the data. The model, hopefully, captures something real in the data. If it does, then it's fair to say that we understand the data better.

But it's frankly a philosophical question what's real or not. No model is going to capture absolutely everything about the thing it models - at that point, it would be the thing. The best we can hope for is that it captures everything we care about.

And no experiment or metric can tell you if you care about the right things. At best it can tell us if we care about a thing given other things we care about. "No cares in, no cares out".

To make it a little more concrete: you could compress a string from back to front. You could build an LLM to help you do that. If you care about file size, that's almost certainly a bad idea, the forward LLM will be better for that purpose. But are there purposes for which the backward LLM might be better? I think that's not so hard to imagine. Often we wonder about "what came before".

reply
A model will not capture every nuance of what it models, but it might capture every nuance you managed to measure properly.
reply
Including the things you didn't even realize you measured, or even conceptualized at all.
reply
I'm not aware of a better definition of "understanding" that would allow me to tell whether some system "understands" some other system. Do you happen to know one?
reply
See: A. M. Turing (1950) Computing Machinery and Intelligence. Mind 49: 433-460.

I mean, my interpretation is that the question Turing tried to answer is equivalent to "How can we determine whether machines understand humans/human thought?"

This only works when both systems can talk about pretty much arbitrary things, but if you want a more general method for less complex systems, perhaps having one system simulate another system is sufficient. (Which is also another Turing invention)

reply
There are many people who would claim that passing the Turing test is insufficient to show "understanding" (compare for example the Chinese Room thought experiment).
reply
Yes but it's (kind of?) a definition as you asked for.

At this point, I am unaware of a better definition. I know the Chinese Room argument (and I disagree with it), but I'm not aware whether the proponents of that argument have a better definition of understanding other than "well, the Turing Test isn't enough"...

---

PS: Interestingly the issue of compression is highly relevant regarding the Chinese Room argument -- the essential element in the Chinese Room argument is that the information is not compressed...

reply