I admit that I don't think formal "papers" deserve their elevated status in today's information economy.
I think they add a whole lot of ceremony that usually results in a format that's harder to understand.
So me calling this a "paper" was a little bit of a dig!
You, being of sound mind, tried the technique and drew reasonable conclusions … but just look at the original post for this blog post: there are a large number of responses by people who clearly don’t understand the technique, missed the experimental weaknesses, couldn’t place the post into the lineage of mech interp results that preceded it … and will leave the discussion with the wrong impression.
In your mind, whose fault is that? Is that that the unavoidable cost of a field, moving too rapidly to engage in peer review?
In its ideal form, the paper lifecycle is not a gate, but a sharpening stone.
I'm not convinced the alternative - delaying publication by 6 months, keeping the new information available only to a much smaller group until then - would have been better.
All of that said, I did just update my own post to include "The paper (OK, the authors classify is as a “blog post”, but it’s paper-shaped) found that frontier models..." inspired by this conversation.