it is what I do to solve hard problems through.
replyeasy stuff happens by itself, but with a system large enough you need a scratchpad and a rubber duck.
One thing about the reasoning is that models are trained to generate a chain of thoughts, but it doesn’t have to be correct, accurate, or reflect the underlying logic of the LLM. It’s the same problem we have with the output, it is something plausible, but not that reliable
replyI do the same thing in my head. There is no underlying logic to an llm. Logic is an external construct alien to human like forms of reasoning.
replyYes, both the output should be "milestones" of sorts, like lemmas and theorems in math. Important plateaus that serve as a launching pad to the next phase. Regurgitating every thought potentially degrades signal:noise ratio.
replyThe hidden states of the tokens likely contain more semantic information than can be extracted by the final projection into token space.
replyActually, how does chain of thought work? Is the LLM actually creating the tokens and then re-reading them, or is the there still a full hidden state under the hood and then the UI just prints that projection?
replyChain of thought seems very specifically like an instantiation of the System 2 thinking of the System 1 and 2 thinking Daniel Kahneman popularized.
replyIs it clunky in that it's a verbalized/languified version of system 2 thinking, and clearly humans do some non-verbal version too?
Some humans apparently only or almost only do non-verbal thinking. It’s called anendophasia.
replySounds like philosophical zombies.
replyMaybe, I wouldn’t know. I just find it fascinating that other people have such a radically different way of experiencing thought. No idea what it’s like.
replyWhat I'd like to see is "brain"-machine interface for LLMs that plugs things like calculator and other modules, directly into the neural network instead of accessing them through in-context tool calls.
replyWhy? I talk in my head and then enunciate only that which is relevant. My speech rate inside is incredibly fast.
replyThe purpose of the scratch pad is very often to refer back to it later.
reply