Why do you need to grep latent space?
replyAs long as it's giving the right outputs, who cares what's in latent space?
If the model thinks in latent space: "God I wish these people would die," and constantly does the right thing, who cares?
Additionally, if one of it's latent spaces that it never explores is a psychopath -> who cares? The path never gets taken...
That's a lot of harmless people walking around with crazy thoughts...
Thinking ‘God I wish these people would die’ could increase its propensity to kill all people, even if that propensity is still vanishingly small almost all of the time.
replyA lot of people are walking around with crazy thoughts. Some of them harm.
Readable reasoning traces are a convenient thing, but they don't have to be true in any way. It's actually dangerous to think that.
replyTell me you never had a crazy thought and you are either a lier or a psychopath.
reply[dead]
reply[flagged]
reply