Conceptually, you should treat them as incremental software instead of magic you grab from others [1]
The killer feature is that coding harnesses tend to have SkillBuilder agent skills so creating them becomes very easy and you can evolve them.
I recommend you build your own for your particular pain points.
Very simple example [2] showing what another user mentioned around "evals" so that you can really achieve good enough correctness for your automation.
- [1] https://alexhans.github.io/posts/series/evals/building-agent...
- [2] https://alexhans.github.io/posts/series/evals/sketch-to-text...
This is an interesting skill plugin for me because I actually face this inverse problem a fair amount where you want to teach people about a repo and the skills associated with it so they understand the intent behind things quickly. Seeing a bunch of skill commands and behaviors doesn’t always make clear why things are the way they are. The people on the other end need context, and the rapidity with which you can create fairly complex stuff means you need a faster way than “three months of onboarding” to get people up to speed.
That's fair but I think this is similar to power tools like vim, obsidian or others. There's the path of grabbing other people's workflows and not being able to modify them to really tailor the tool to your needs and there's the minimal incremental path that empowers you and gives you control all the way through. It gets you to understand the tools and you'll be able to think possibilities that match your exact problems.
I'm not dogmatic about it but I do really recommend it. You can see the transformative shift once people start "skill building" instead of "skill consuming".
Edit: The approach I mention works with non engineers/developers. So there's no different technical bar.
also you can secure/lockdown tool calls better and make the agents tasks retryable, give it failure modes etc. (not if ur laptop dies during agent work its only god and the agent who know what happened to your code.. oh no wait. the agent needs to just spend 100k tokens to remember where it was (great way to spend ur money).