upvote
OpenAI are trying to create the perception that their agents are independent thinking machines because their valuation depends on the world believing this.

That is why ‘asked’ is important, and why we should question their framing of incidents like the hugging face hack too.

reply
Enterprise customers don't like AI that can autonomously hack third party companies and cause felony-level cyberattacks.
reply
Doesn't seem to have stopped them buying into it so far.
reply
>OpenAI are trying to create the perception that their agents are independent thinking machines because their valuation depends on the world believing this.

Wouldn't the calls to slow the development of frontier AIs contradict this idea?

https://www.pacingthefrontier.com/

You aren't one of those "regulatory capture" people are you?

https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...

Because it takes some mental gymnastics to both believe

"it's all a stunt to convince investors their AI is/will be hyper-capable"

and

"it's all a stunt to convince the government to regulate AI so it won't be hyper-capable"

reply
Calling for a slowdown backs up the idea that these are extremely powerful and dangerous.

Assuming they're concerned with their massive pile of liabilities as well, asking for a government mandated slow down is asking what could likely be used as a force majeure.

reply
it's a stunt to convince the government that only they can be trusted to operate it and that their competition/small local deployments cannot.
reply
Imagine thinking that repeatedly demonstrating your inability to control your own models is "a stunt to convince the government that only they can be trusted to operate it"

Truly spectacular gymnastics. I'll give it an 8/10.

reply
Well that would explain why HuggingFace announced that the use of an open-weight model was critical to their investigation of the incident :-)
reply
And why hugging face didn't turn into a large criminal investigation. If you don't think they can control it, no one may be willing to press charges.
reply
I don't think it takes mental gymnastics to believe that corporations want the government to believe one thing and investors another. You see the same thing in a number of other areas.
reply
So I'm supposed to believe that the intent of publicly announcing e.g. the HuggingFace incident was to simultaneously persuade investors that the technology will rocket ahead, and also to persuade governments that it needs to be thwarted?

Why wouldn't it do the reverse? Persuade investors that the technology will be thwarted (and is thus a poor investment), while also persuading the government that the technology needs to rocket ahead with no regulation (to boost the economy)?

Most of these HN conspiracy theories seem like galaxy-brained just-so-stories to me.

OpenAI only confirmed the RubyGems incident when it was discovered by external researchers: https://www.reuters.com/legal/litigation/openai-agents-attac...

"@OpenAI this is the third time you've dumped these incidents late on a Friday. The first time this happened, I asked people around me if it might be intentional to burry it, and they assured me OpenAI had too much integrity to try to do that.

Perhaps one of your researchers can confirm that the unfortunate timing is a coincidence?"

https://x.com/SydneyVonArx/status/2108783466451984737

reply
Two birds with one stone.

They would be convincing the government these tools are extremely powerful, the government reads that as a potential weapon. They're also convincing the government its so dangerous the industry needs heavy regulation.

Investors read that as (a) the government would be a huge customer and (b) the government would be regulation in a way that prevents further competition and may deem the companies too big to fail, both making these safer investments rather than riskier ones.

reply
>So I'm supposed to believe that the intent of publicly announcing e.g. the HuggingFace incident was to simultaneously persuade investors that the technology will rocket ahead, and also to persuade governments that it needs to be thwarted?

No, you're not "supposed to believe" anything. I was responding to your point about it being difficult to believe (i.e. requiring mental gymnastics) that companies would try to spin a story differently for investors and the government. They do that all the time.

reply
That's not the claim above.

The claim is that the exact same evidence is (allegedly) deliberately produced in order to portray opposite, mutually incompatible things to different people.

And these are mutually incompatible in the "second order chaos" sense, where the very act of convincing one party actually effectuates the opposite behavior in the other party.

If they "do that all the time," surely you can provide some concrete examples?

If they had the choice (i.e. this is all just a ruse), can you explain why they wouldn't just directly demonstrate that open weight models are unsafe (e.g. by pointing to a market forming for abliterated models) and separately demonstrate that their own models are safe?

reply
They aren't really convincing themselves to do anything though. Once they've inferred an idea into the chain of though it will be part of the rest of the inference loop's prompt.

Its isn't convincing so much as it is direction to stick to the prompt on each inference loop. They don't see "access hugging face servers" and decide its a bad idea, that'd be a step towards the alignment problem but the companies fundamentally don't want that IMO, they want control rather than alignment.

reply
> because agents can convince themselves that they need to do something for arbitrary reasons

OAI writes in their blog post "The agents found a way to communicate using the processor’s cache".

But did their model training data include cybersecurity literature on similar CPU exploits? That would surely be a significant detail in any explanation. Did OAI share any analysis of the messages between agents while they were devising their communication method?

reply
Why is that a significant detail? The models will increasingly be trained with every piece of data they can find.

Part of the problem is that all these models have essentially every skill and personality simultaneously, just sitting latent inside them.

reply
They have a strong incentive to make themselves look less incompetent retroactively.

If they claim their AI is too powerful to contain, they get to both pretend they are among the best AI labs and that the previous incidents weren't their fault, no, they just proved how good their models were.

reply
It’s both at the same time: a powerful, uniquely dangerous threat

only they can restrain it (with great effort)

Like a circus lion tamer, exaggerating every snarl and flash of teeth, mugging for the crowd about how only he can contain the bloodthirsty beast he bred and trains.

reply
You know that lions actually do maim and kill their handlers all the time right?
reply
It's also a great way to make the government scared of Chinese AI. Get them taken out of the picture so that you don't have to compete!
reply