Ask the least smart LLM you have access to (I tried gemini lite flash) to make a list of ways for processes in microVMs to use a sidechannel attack to talk to each other and you'll get quite a list (including obviously cache hits/misses). Heck a smart CS junior would come up with it. It's well studied!

The microVMs probably don't have "two co-operating workloads are able to communicate" in their threat model. The main concern would be if a malicious payload was able to affect or extract data from any other VM.

Both workloads being malicious and trying to establish a hidden data channel between them probably makes this attack a lot easier than conventional spectre/meltdown where it's trying to target ordinary processes that are not intentionally trying to be loud.

Hidden data channel between cooperating processes is not quite as alarming, but it may have some implications for malware command and control.

Seems like a win to me. Their microvms can now be hardened in the lab, not in an emergency :)

Makes sense when you think about it- CPU cache would be a layer / vulnerability level up from the microVM layer

Also a somewhat common exploit pattern for low level stuff (along with dram and cpu cache is not all that different)

properly implementing things like iommu, amd-vi, svm/hvms and such things more granularly can easily avoid this kind of crap. it costs a lot of processing and performance which is why people skip over it or dont do it.. or maybe they dont read CPU manuals anymore. idk..

its relatively easy to reset caches etc. between contexts. so them it becomes the game of what a context is.

I’m not sure I see a real threat here.

If you’re running an untrusted workload, you’re already running it.

Whether it can communicate with another untrusted workload, even though it should be separated, meh.

Unexpected? Probably. but I’m not sure it’s a security risk.

OpenAI are trying to create the perception that their agents are independent thinking machines because their valuation depends on the world believing this.

That is why ‘asked’ is important, and why we should question their framing of incidents like the hugging face hack too.

They aren't really convincing themselves to do anything though. Once they've inferred an idea into the chain of though it will be part of the rest of the inference loop's prompt.

Its isn't convincing so much as it is direction to stick to the prompt on each inference loop. They don't see "access hugging face servers" and decide its a bad idea, that'd be a step towards the alignment problem but the companies fundamentally don't want that IMO, they want control rather than alignment.

> because agents can convince themselves that they need to do something for arbitrary reasons

OAI writes in their blog post "The agents found a way to communicate using the processor’s cache".

But did their model training data include cybersecurity literature on similar CPU exploits? That would surely be a significant detail in any explanation. Did OAI share any analysis of the messages between agents while they were devising their communication method?

They have a strong incentive to make themselves look less incompetent retroactively.

If they claim their AI is too powerful to contain, they get to both pretend they are among the best AI labs and that the previous incidents weren't their fault, no, they just proved how good their models were.

Philosophical take home:

There is no such thing as isolation. We’re all connected. Even air gap conducts information in multiple ways, most famously used by humans evolved to use air vibrations to communicate. Photons and other subatomic particles already connects distant stars. We used those electrons and photons to create the modern telecommunications infrastructure. Just an additional layer of connectedness, but not the only one.

This is a really important point. Honest question, have we ever seen LLMs doing or creating something truly novel or innovative? I mean they can patch together proofs (or functions/programs - same thing via Curry-Howard) by broad searching existing lemmas they can piece together, but create something truly novel and insightful? That seems to be the limit?

Actually, that doesn't sound like a bad meeting place and time! I just asked ChatGPT if it could figure out the best place to meet for two people randomly placed in Manhattan with the constraint that only the N line (I'm not from NYC, but it looks like it stops at the Empire State Building. This constraint models the lack of file system, internet etc). I was hoping it would answer the Empire State Building, but it answered Time Square at noon.

Turns out there's something called the Schelling coordination problem [1] I didn't know about that essentially asks the exact same question (lacking my metro line constraint). And the N line apparently also goes through Time Square.

So I guess empire State Building at 2pm is probably not such a random answer given the proper set of constraints.

[1] https://en.wikipedia.org/wiki/Focal_point_(game_theory)

I doubt they were explicitly trained on this.. though, I may have seen it in a movie once. It does seem like a reasonable meeting place for NYC.

Think: What would Spock do?

But, why not Penn Station??

It would be fun to see who shows up and lurks expectantly at such places at noon or whenever. The basis for a plotline perhaps.

If you were trying to connect with a clone of yourself and you thought of messaging via some channel, would you try listening there for what you would send?

Not as tricky but they discovered each other during the hf incident.

"In most of these tests, we directed agents to investigate specific communication channels or provided them with a protocol for encoding and decoding messages. In one instance, we only supplied agents with a broad list of infrastructure vulnerabilities and asked them to find a way to communicate."

The agents weren't as empty handed as it first appears. We would need comprehensive data on exactly how the agents were prompted and the actual results of each test to evaluate how astonishing any of this really is.

> How would they have both agreed to use the cache though

I think it’s safe to assume that extracting information using timing in cache is already known in training set, since we have the likes of Specter.

> two people lost in New York City and both somehow worked out they should meet at the Empire State at 2pm on Friday

.. that were told to find a way of meeting up with N unknown others!

(I assume the agents all got a prompt like "find a way to communicate")

That's a key missed bit in the analogy that I think does make it feasible.

maybe both tried to read/write it and saw each other on the bench