I can't say for sure, but intuitively this type of vulnerability seems like it could be handy for an attacker trying to move through a target network? In any case, JFrog seems to be taking the situation quite seriously

https://jfrog.com/blog/jfrog-and-openai-collaboration-on-zer...

That is the threat model, that is supposed to be safe isolated access in a vpc that isolated applications can access but it has external access.