It gets tricky when you want to say “allow this particular tool to have a little more access.” Like allow the codex client to access the GPT API but not some random other site the LLM learned about. Or like letting git write to .git to commit changes but don’t let something install git hooks. Or allow downloading certain packages/versions from a site like PyPI, or allow certain operations on certain GitHub repos in a way that doesn’t line up with how you can configure an access token.