upvote

points

by lpapez14 hours ago |
comments
upvoteby jychang14 hours ago|
next
[-]
It's also literally factually incorrect. Pretty much the entire field of mechanistic interpretability would obviously point out that models have an internal definition of what a bug is.

Here's the most approachable paper that shows a real model (Claude 3 Sonnet) clearly having an internal representation of bugs in code: https://transformer-circuits.pub/2024/scaling-monosemanticit...

Read the entire section around this quote:

> Thus, we concluded that 1M/1013764 represents a broad variety of errors in code.

(Also the section after "We find three different safety-relevant code features: an unsafe code feature 1M/570621 which activates on security vulnerabilities, a code error feature 1M/1013764 which activates on bugs and exceptions")

This feature fires on actual bugs; it's not just a model pattern matching saying "what a bug hunter may say next".

reply
upvoteby mrbungie12 hours ago|
parent|
next
[-]
Was this "paper" eventually peer reviewed?

PS: I know it is interesting and I don't doubt Antrophic, but for me it is so fascinating they get such a pass in science.

reply
upvoteby ACCount3711 hours ago|
parent|
[-]
Modern ML is old school mad science.

The lifeblood of the field is proof-of-concept pre-prints built on top of other proof-of-concept pre-prints.

reply
upvoteby emp173446 hours ago|
parent|
[-]
Sounds like you agree this “evidence” lacks any semblance of scientific rigor?
reply
upvoteby mrbungie4 hours ago|
parent|
[-]
(Not GP) There was a well recognized reproducibility problem in the ML field before LLM-mania, and that's considering published papers with proper peer-reviews. The current state of afairs in some ways is even less rigourous than that, and then some people in the field feel free to overextend their conclusions into other fields like neurosciences.
reply
upvoteby Jensson4 hours ago|
parent|
prev|
next
[-]
> This feature fires on actual bugs; it's not just a model pattern matching saying "what a bug hunter may say next".

You don't think a pattern matcher would fire on actual bugs?

reply
upvoteby emp173446 hours ago|
parent|
prev|
[-]
Mechanistic interpretability is a joke, supported entirely by non-peer reviewed papers released as marketing material by AI firms.
reply
upvoteby pousada13 hours ago|
prev|
[-]
Some people are still stuck in the “stochastic parrot” phase and see everything regarding LLMs through that lense.
reply
upvoteby windexh8er12 hours ago|
parent|
[-]
Current LLMs do not think. Just because all models anthropomorphize the repetitive actions a model is looping through does not mean they are truly thinking or reasoning.

On the flip side the idea of this being true has been a very successful indirect marketing campaign.

reply
upvoteby pousada9 hours ago|
parent|
[-]
What does “truly thinking or reasoning” even mean for you?

I don’t think we even have a coherent definition of human intelligence, let alone of non-human ones.

reply
upvoteby chpatrick6 hours ago|
parent|
[-]
Everyone knows to really think you need to use your fleshy meat brain, everything else is cheating.
reply