undefined

points

[-]

Mechinterp in general is just completely undervalued right now (and agreed Anthropic's team is doing the most rigorous work, now accompanied by Goodfire). They're doing the closest work to neuroscience's in vivo 'thought-tracing', which is just the most wild science fiction sort of thing to be working on, and yet I feel the average person has no idea this sort of work is happening. When combined with the idea of the 'universal subspace hypothesis' (explored under the paper of the same name), you really start to bridge the gap from engineering to something more philosophical and spiritual. But I digress...

by gom_jabbar1 hours ago|

parent|

[-]

Haven't heard about the universal subspace hypothesis yet, so I appreciate the digression.

by thoughtpeddler45 minutes ago|

parent|

[-]

Ya, super interesting research area the authors explored of basically trying to answer the question: "Is there a canonical/intrinsic way that concepts/representations/information are 'stored' in the universe/reality?".

They tested that by performing "spectral analysis of over 1100 models - including 500 Mistral-7B LoRAs, 500 Vision Transformers, and 50 LLaMA-8B models ... by applying spectral decomposition techniques to the weight matrices of various architectures", and concluding that "deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces", showing that "neural networks systematically converge to shared spectral subspaces regardless of initialization, task, or domain".

Not just philosophically interesting but also has practical implications for being smarter about how to reuse models, model merging, developing more sustainable training and inference algos, etc.

Paper source: https://arxiv.org/abs/2512.05117

by janussunaj1 hours ago|

prev|

[-]

Did you also talk about "head and shoulders" and "pennant" patterns in stock charts? Or where the "smart money" is at? I'd like to subscribe to your paid newsletter.