No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.
> Regardless, if all you want is a next-token selector, save your money and roll a die.
Tell me, where is the emergent symbology guiding that dice?
The paper claims no symbolic richness beyond that evident from the undisputed next-token prediction.
> Tell me, where is the emergent symbology guiding that dice?
There's none. That's my point.
I mean, of course they are, that's literally what the inference loop does. You can look at the source of your favorite model runner and you'll see exactly that.
What I find misleading about this term is that it focuses attention on the "next token" part and glosses over the "prediction" part as some sort of unspecified "statistical algorithm" - even though this is where most of the work happens and where the interesting questions are.