Perhaps you can also help me understand why #1 being right and #2 being wrong would be some win for alignment that people on X claim it to be?
replyMy impression, being a total outsider, is that the reaction feels like a bunch of insider bros hyping up their insider bro friend. It feels more like PR than a genuine groundswell behind a truly revolutionary idea.
Because if each neuron represents a small set of similar concepts instead of several completely different things, it should be easier than expected to look inside the neural net and see what it's really thinking.
replyIt might still be quite difficult since the network is huge, and just because you know what the individual neurons represent doesn't mean you understand the complicated thoughts they're building. But easier than expected is still good news.