I see this dilemma with LLMs all of the time.

Should you use the LLM to do the thing directly, or use the LLM to implement a tool that does the thing?

I tend to reach for the latter, it’s easier to reason about.