upvote
Yeah, it's been known for a very long time. Richard Feynman alluded to it in his speech The Value of Science [1] where he discussed a Buddhist proverb:

  To every man is given the key to the gates of heaven; the same key opens the gates of hell.
He then goes on to say:

  What, then, is the value of the key to heaven? It is true that if we lack clear instructions that determine which is the gate to heaven and which is the gate to hell, the key may be a dangerous object to use. But the key obviously has value: how can we enter heaven without it?
[1]: https://calteches.library.caltech.edu/40/2/Science.pdf
reply
> The hilarious part, though, is that it's not the AI that's working around the rules. That's the scenario that's been in science fiction, but it's not what's happening. It's the human users making use of our agency to get the AI agents to work around the rules. Despite calling them "agents", current AI agents don't seem to be able to that particular something. Yet, at least.

Well, yes. Until people are putting the LLMs into actual mechanical robots, "agency" boils down to flipping bits in memory or storage (even if they're ones that humans consider really important, e.g. because they represent a bank ledger) or convincing humans to take action. One can only "work around the rules" to the extent that one can "work".

But even in Asimov's books, at least some of the scenarios involved humans misleading the robots to use them as pawns in a greater scheme.

reply