This is a "they have guns so we need guns" scenario.
replyYou can't guarantee everyone else will use a neutered model.
Isn’t this essentially what anthropic is doing, albeit in a manual fashion? They work with code owners to run mythos and find issues.
replyCode owners - no. Ultra wealthy code owners with connections, and a few peasants with popular projects, for public image.
replyOnly if you're some big corporation with deep pockets. They actually accepted me into their cyber program but Fable's still locked down.
replyOpenAI now makes it easy to join their verified security program. Took me 5 minutes, and I was able to get GPT to do a full end-to-end pen test
replyIs this about the new project Blue and Red thing? Or just pre-existing Trusted Access for Cyber program?
replyI've joined TAC, but still have to dance around it.
Impossible with source code, possible to bypass with app/site
replyDon't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.
replyWith Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify.
replyIn this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.
Yeah, for web apps, you can trick models by simply proxying it and pointing the models to that localhost. They then think they're not working on a live target.
replyHave personally tested this with Opus and Sol and it works.
Classifiers are tricky though. Here's where open weights will win.