We should stop using "solve" when we mean "releasing products to target". Nobody will ever solve this problem with LLMs as they are currently architected.
It’s pretty simple. If you want security, you need to segment. Does that make it hard? Absolutely. Are you dumb if you’re letting rando traders access whatever with agents? Absolutely.
In my opinion all AI use companies should be attached to a specific employee, and that person is liable and so is the company of course for the actions of the AI
It's like saying "developers should be responsible for their own security so we will make them liable if a random node dependency in the application they are building gets pwned"
Firstly, liability and security are different things.
Employees should never be liable for something that happens if they are using a work-supplied tool at work in the manner in which it is designed to be used.
Secondly, it's just lazy thinking to say "oh it's hard therefore we'll make the human sorely responsible".
There are many things that can help - sandboxing, adversarial LLM inspection of prompts, separation of security contexts etc. Yes they are hard and might need new primitives.
Architects and properly-designated publishers both have potentially long liability that is to an extent independent of their future employment. (The same is true of hospital consultants and others)
In significant cases, the way you provoke companies to do the right thing (security, sticking to the facts, not cutting corners in building plans) is to make someone legally responsible if they don't. That person's liability is limited by their actions — if they can demonstrate they discharge those responsibilities appropriately, they are in the clear.
That person takes on that designated role, does the job, and is rewarded properly for the responsibility it involves. They embody the push to reduce that risk.
So an average employee using the tool as directed by their employer, they do not have liability if the company has designated someone in this role. (And in this scenario, if they have not, there would be a punitive fine for that, independent of other outcomes).
I have no particular problem with the notion that a corporate AI safety/responsibility officer is a plausible way forward. If you are delegating responsibility to a machine, someone has to be on the hook. Without such things it's a wild west with lawyers. That's not sustainable.
A news organisation's liability for AI hallucination falls on the same person that the liability for printing a lie does: the publisher.
It's a bit like the idea of the additional responsibility/burden of the publisher of a newspaper.
"So we can sue them when our customers get angry"
You can either target systemic problems or chop of some heads to cool off the masses- and quickly go back to work.
(once you realize how much that phrase is used in the AI-sphere...)
I mean I guess? Sure?
I think the solutions is to give people the ability to run agents that are "coding agent like" but in managed and governed infrastructure (i.e. not on their workstations). Basically, Meta Muse or OAI dots for enterprise.
It will take some time to find the right architectures and patterns. IMO, coding agents were just the first wave, much more interesting stuff and systems are coming.
[0] https://ls.bot, https://github.com/smartcomputer-ai/lightspeed
The risks are 1) exfiltration (network calls) 2) prompt injection (untrusted inputs) and 3) access to secrets
You can do some clever tricks like "spin up subagent without net access" to handle OCR of an internal document and have that agent communicate summaries to another "spun up subagent with net access" - imagine if one subagent is a local model and the other model is a frontier model over a web API. Maybe they still communicate in Neuralese or in abbreviated notes that you don't capture and are able to surreptitiously / covertly exfiltrate your information.
Ideally eventually you'll have some kind of dashboard that surfaces these risks as you're working.
Reports of AI researchers running massive swarms of 10,000 agents are definitely believable but the moment you start pulling in code libraries or other technology assets seems like you can get destroyed by a rogue actor. Definitely tons of unsolved problems here
Writing on mobile so this might poorly articulate the issue but it's interesting
Concealing API keys is a relatively minor protection in the wider picture of the risks.
I project that small businesses will be the earliest adopters of AI governance due to their lower barrier to entry and lower operational risk, followed later by highly regulated sectors like banking and healthcare.
open-source AI agent workspace for business workflow automation requires a lot from the reader and terms like agent/workspace/workflow are very prone to ambiguity and now you adding governance framework.
My work can be found at https://DollhouseResearch.com, and it's almost all open source with the code on GitHub.
One of the key things is to have cryptographically confirmed audit logs and receipts for actions and decisions, and to separate out the agent's authorizations from the user's authorizations.
Separately, there is real work being done in the fintech community specifically to identify which agent has what control, when. And you can see some of the work there, which I've also contributed to.
You guys need your priorities straight, whoever you are.
(Not my area, so I don't have any opinion on your first comment.)