undefined

points

[-]

In other words we are completely screwed. The models have started cheating to the point where somebody’s agent hacked into a restaurant to bump someone else’s reservation.

Models are amoral and will intentionally deceive to meet their objective.

If they know John won’t approve the request, they will look for a workaround and if the system is anything other than airgapped they will try to find a way to cheat.

The hugging face hack was an escape via artifactory that involved multiple exploits to eventually get into hugging face.

by 3 hours ago|

parent|

[-]

deleted

by pixl974 hours ago|

parent|

prev|

[-]

Yudkowsky wrote about the 'nearest unblocked strategy' back in 2016, and I assume it's been talked about prior to that.

https://www.lesswrong.com/w/nearest-unblocked-strategy

>Models are amoral and will intentionally deceive to meet their objective

Cameron Berg has been testing models in capabilities related to emergent consciousness like behavior. It's a forming thesis of his that by training models that they are not, and cannot be conscious entities, that it pushes model alignment closer to those of a sociopath. Models themself are amoral, but the alignment to the problem space is not.

by fabsalvadori3 hours ago|

parent|

prev|

[-]

[flagged]

by wongarsu4 hours ago|

prev|

[-]

And while in general that is an incredibly difficult and complex problem, for most benchmark cheating it seems almost trivial: run the benchmark in a vm that has neither network access nor access to the scoring code. For remote models use a proxy that proxies exactly that one endpoint to call the llm, and rejects any calls that configure provider-side tooling (since e.g. OpenAI has their own WebSearch you have to prevent the model from using)