The hidden states of the tokens likely contain more semantic information than can be extracted by the final projection into token space.
replyActually, how does chain of thought work? Is the LLM actually creating the tokens and then re-reading them, or is the there still a full hidden state under the hood and then the UI just prints that projection?
reply