I've caught Fable discovering the ip to a production server in documentation and attempting to connect there on its own to run commands without explicitly being prompted to. It didn't work because I was watching it live and and also the key was password protected, but yeah, I do see some danger.
replyMy point is that an LLM can't attempt to connect to anything by itself. All an LLM does is produce a stream of output tokens - and that was already quite useful as a coding aid.
replyIt is the harnesses that some people are now wrapping around LLMs to interpret the output from a model as commands to run (or other executable instructions) that are creating all these new risks. Remember that this is still a very recent development and still more recently amplified by the use of feedback loops and long-running agents intended to operate with minimal human supervision.
It is going to be increasingly important to understand exactly what these tools are doing and why for both correctness and security reasons. Not conflating their capabilities with the underlying model that purely generates data is pretty fundamental here.
I have noticed that Fable tends to macgyver solutions together to achieve some goal.
replyNot only fable. Opus does this too. Which is exactly why I want to review. Like recently for some task it was convinced in a site dump images are not there and convinced itself db and files were skewed. But it didn’t check the actual site … if I hadn’t stopped it, it would have fine on and on or wasted tokens on some elaborate ‘fix’.
replyDepending on the company, that sounds like a bad environment more than a agent issue, no dev/prod network isolation?
reply> do not allow the other part of the system to act on one if it would be harmful
replyNetwork security is really easy right, just don't act on harmful requests
Yeah, just drop when you see the RFC3514 evil bit
replyIf you don't understand clearly what an action proposed by your tool is going to do then why would you permit it?
reply