Half of these results are just Astra being able to reason through mathematical operations like matrix multiplication and the other half are fairly straightforward applications of mechanistic interpretability.
replyCould it be already baked into these models because they have already been trained on this task this way?
replySomething like mirror neurons.
replySome structures in Astra get activated by the statistics of the Qwen weights, and these structures are associated with "breakout"
My guess: There is a ton of data online of just plain weights laying around. No human would ever actually read it. LLM's read it all during training.
replyBut that would cover finite set of cases. I believe this result is more general.
replyI mean it would make sense if the training using that data resulted in a general set of equations/patterns, it's still really surprising though...
replydeleted
replySee my other comment in this thread
replyBased on our current understanding of LLM's, it shouldn't be.
replyThis sort of throws a wrench in the whole "stochastic parrot" idea...
I am not surprised, though I expect many to be.
Yes, this is hardly a shocker to anyone who used the latest models to convert/optimize AI models.
replyThe other day I was thinking about the definition of AGI.
I was asking myself the question that when there is no form of data that I can think of throwing into Claude Code and have it not come up with some sort of intelligent answer about it, is that already general intelligence?
Others have though much deeper about this its all documented in academic papers and books
replyIMO yes
replygeneral knowledge perhaps, but certainly not intelligence in any traditional sense
replyAnd yet you ask it 3 times to not do a thing, and it does it in a 4th way. A human doesn't fail in this way. And I'm not talking about memory here, single context.
replySo yeah, it can do everything, but fails in trivial ways around the doing part.