> The harness instructs them to behave this way. Also this approach saves tokens. The scripts allow to edit files in bulk, and most of the session cost is in cache reads (e.g. for 300K context each command costs the same as 30K input tokens).
replyI understand the reasoning, but at that point wouldn't the LLM be better off creating `sed` commands and executing those? I mean, if it's already executing Python, it can literally do anything to the environment, so using `sed` is at least as safe, with a bonus that it (or a subagent, or a human) can double-check the intention with the sed script and flag incorrect or missing changes.
I've experimented quite a bit with giving agents python vs sed + awk. They make mistakes with both, a lot. The only thing that has stood out is that agents reach for python too quickly if it's available, and that awk causes the least problems, while sed might take several attempts to get results, similar to python.
replyAlso it's the only way that makes sense when you need to work with big files, or large amount of files, or documents that look small when fetched through a RAG tool, but then you read one and get hit with couple megabytes of base64-encoded binary data you didn't expect because RAG tool stripped out embedded images...
replyAsk me how I know. Or don't. I have a standing rule for all agents warning about that failure mode (and related, doing `ls` in `/tmp` and few other directories that like to accumulate files by the hundreds..)
My harness forbids it, they end up spending time debugging their scripts
reply