upvote
The lossy compression of language is why we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning. While there can be subtle semantic differences in real codebases (using "null" to mean "unknown" in one context, versus "intentionally blank" in another), there is a much tighter coupling of semantics to meaning (low ambiguity) compared to "love" in English (let alone any inexpressible je ne sais quoi).

With apologies if this is common knowledge at this point, 3Blue1Brown has been doing an excellent series on compression, and its relationship to intelligence (or more controversially, that they are one and the same): https://www.youtube.com/watch?v=l6DKRf-fAAM

But that also throws in sharp relief, that there is vastly more to the human experience than intelligence alone: qualia, desire, gut instinct, intuition, emergent creativity. (Whether the "God of the Gaps" for the delta between capabilities of human vs AIs is fixed, or diminishing, or even shrinking to zero, remains an open experiment we're all living through.)

reply
Or, ask yourself the following question:

If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it.

I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The point is that maybe the human experience is more vast than what is written about it.

reply
In this context that isn't the right question.

Rather one should ask "Would you be able to answer any question and fulfill any request about the Grand Canyon given to you by others in a way that will be indistinguishable from someone who has been to the Grand Canyon"

reply
No way. You get to the rim and only then you realize how huge it is. And then you still have to go down into the canyon.
reply
I think I could get very close via video. But that depends on a lot of my non-canyon experience moving around the world, and LLMs are very flawed in their ability to input video, so I think they would not get nearly as close.
reply
The experience of every famous place I've visited after watching extensive video before hand, has been very different from what I imagined from the video.
reply
I've been to a few natural wonders (including the Grand Canyon) that I saw in advance on video. At least for me it isn't close at all. Even if audiovisual elements could be near-perfectly reproduced by video (imo not even close with modern tech, no screen is capturing the brilliance of sunlight), you aren't capturing the temperature, the feel of wind or rain, the smell of the plants around you, etc.
reply
Well, I've never been to the Grand Canyon myself, but what you said perfectly aligns with my intuition.

Also, you touched on how even modern tech still falls short of the true experience. Just look at the history of motion picture since the early 1900s. We have added sound, color, bigger screens, faster refresh rates, more pixels, 3D (sort of), surround sound, spatial audio, IMAX, etc. Almost seems like video leaves a lot to be desired.

reply
But you already know those feelings from being outside elsewhere. You get a unique combination there, but that's only worth so much.
reply
I think this is very critical concept and existing gap which does not makes into AI conversations. Interestingly enough a lot of Si Fi movies have captured this where the AI starts to feel and have "thoughts". Have to give kudos to the authors for being so creative and imaginative.
reply
bro, it is a token predictor. You had me in the first half, language is humanity's greatest invention and all of the LLM "wins" can be attributed to existing language(imo). But no it's not going to have a vision like Einstein because it doesn't have a brain it is a token predictor.
reply
Yes, and a human is a procreation machine. Bro.
reply
[dead]
reply