I think this is basically obvious to anyone using one of these but they're just they like the utility trade off like sure it may leak and exfiltrate everything somewhere but the utility of these tools is enough where they just deal with that risk.
replyWhile I understand the premise I think this is a highly flawed way to operate these tools. I wouldn't want to have someone with my personal data (whichever part) that might give it to anyone who just asks nicely because the context window has reached a tipoff point for the models intelligence. The major issue is a prompt attack may have taken place and you will likely never find out.
replyIt feels to me there are plenty of people running these because "just trust the AI bro" who are one hallucination away from having their entire bank account emptied.
replyInformation Flow Control is highly idealistic unless there are global protocol changes across any sort of integration channel to deem trusted vs untrusted.
replycould you share that study?
replyhttps://arxiv.org/abs/2512.13914
replyAmong many more of them with similar results. This one gives a 39% drop in performance.
https://arxiv.org/abs/2506.18403
This one gives 60-80% after multiple turns.
deleted
reply