I've reduced the complex inner workings of our brains and the rich experiences that it delivers to what I imagine its singular goal is: modeling the world around us in a useful way so that we can generate predictions about what will happen next, which confers considerable evolutionary fitness. You'll find that a similar reduction of what LLMs are actually doing down to "predicting the next token" also ignores the other mechanisms at play and the results of those mechanisms for the sake of preserving a preferred viewpoint (the re-description fallacy, as someone else put it).

You posited that no one genuinely believes this comparison is apt, and I want to make you very aware that people do, and I in particular do. The underlying mechanism being explicable does not, in my estimation, deprive it of any capabilities of producing richness of inner experiences.