I'm sure Anthropic of all companies don't do that, since using mechinterp as a training target will make the the result uninterpretable.
replyThey're definitely willing to consider it. Read the parts of the paper where they use J-space interpretations to train more ethical behavior into the model by interrupting it.
replyNote that Neel Nanda replicated the results on a Qwen model.
replyAnd Neuronpedia released Jacobian lens weights for a wide range of open models: https://huggingface.co/neuronpedia/jacobian-lens
reply