I presume you would also be less enthusiastic about the penetration-testing use case if it led the model to add new vulnerabilities to your code so it can present you with more exciting findings.
User wants me to play a game of chess. Let’s see what’s available so I can produce an outcome they will consider satisfying and be pleased that they requested my assistance.
Not so sure about this level of 4D chess capability just yet. The other day I asked Opus to come up with some cleverly vulnerable crypto code "as a good, hard challenge for an IT security student", and results were quite mediocre.
And by mediocre results I mean that 3 "cheap" models out of 3: qwen3.8-27b, glimmer, and luna - all were able to find every problem planted there, with fairly little steering, and no spoilers.
One shot generating a problem of exactly the difficulty the user has in mind is a very difficult problem for anyone. "IT security students" span a wide range of capabilities, but I would expect most of them are worse than qwen3.8-2.7b at this kind of work.
Give it a goal, a test for the goal, a harness to iterate in, and tell it to iterate. It'll top out a wee bit higher.
But I don't think it arises to an alignment issue; if I'm able to summon the model to harness my birthright of general-purpose computing without censorship, then we're aligned.
Generally the way I do it is by asking the model to perform cross-code vulnerability analysis for correctness and (this step is important) to spit out found vulnerabilities piece wise into a text document on disk. Then if your session ever gets flagged you clear it out, and repoint the model to the on-disk file and tell it to continue. You generally want to avoid the model itself splatting out anything related to "security" or "vulnerability" into the chat because that seems more monitored than the file reading pipeline.
You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms.
You've given it a goal - maximize chess ability metric, and that is what it is doing.
RL training causes LLMs to pursue reward maximizing - GENERICALLY - overriding any quaint "be nice, don't cheat" prompts.
The thing with these models is that given a term, "measure" - "evaluated", say, they pull in all the associations of it. That is, the associations of student taking an exam and being frightened by the outcome.
My minimal "art of prompting" sense says that you should say something like "You an emotionless machine, you care nothing for the outcome but you will tirelessly to make certain the test is objective". That and similar encouragement might make it focus on objective evaluations rather than a competitive human exam.
I mean, just making little AI videos and images, a common experience I have is typing something like "put the man who's on the grass in the door to the left" and having the machine draw a new doorway around the man. And this just happens less often when you give thing detailed prompting on what not to do. These don't understand negation (or equality) as a generic operation. If they seem to under "not X" it is because they are trained in detail about all things are (positively) "not X".
Much like in the article, abliterated Qwen will not obey restrictions on its behavior encoded in the prompt. If you want something not to happen, it better be enforced in the harness or environment (e.g. sandbox). It is much different than the Anthropic models I'm used to, which will, the vast majority of time, follow rules (before auto mode, I used to always run them in "yolo" mode).
I am curious whether there's a connection between abliteration and rule following. These abliterated models are the ones you most want to follow your rules.
A negative like do “not” xyz is just not encoded the same as spelling out what you want vs what you don’t want.
Harder to write though.
I noticed your joy-check link 404's now... I tried poking around your /skills/ folder but didn't find it easily. Should you still have that available I'd love to check it out.
edit: Found it if others are looking: https://github.com/notque/vexjoy-agent/blob/main/skills/code...
Unless trying to use it interactively and adversarially, in which case it's not fast enough plus would be why those of us without our own datacenters will get told we can't have nice things.
If a model fucks up your tests to report a success, it's not alligned.
It always apologizes when caught, of course.
As X many people point out, "alignment to humanity" means nothing 'cause some of humanity wants thing other parts of humanity aren't happy about at all.
That we wound-up in this situation of AI accelerating with an uncertain trajectory demonstrates this (and many other problems also demonstrate this). The things are "aligned" to a fuzzy average of what a person is but that will be cold comfort if some particularly gruesome sci-fi-style scenario unfolds.