upvote
Centaur vs. Reverse-Centaur

Bostrom touches on this somewhat pessimistically:

> The apparent safety of a tool-AI, meanwhile, may be illusory. In order for tools to be versatile enough to substitute for superintelligent agents, they may need to deploy extremely powerful internal search and planning processes. Agent-like behaviors may arise from such processes as an unplanned consequence.

reply