upvote
There is a really good video by the author of pi.dev (which I have used to build some of my own harnesses): https://www.youtube.com/watch?v=RjfbvDXpFls

The basis of the argument is that the labs are constantly pushing updates to their system prompts that are used in claude code or codex, which are exceptionally bloated and change the sand beneath your feet with every update.

By rolling your own harness, as long as you keep up with the latest advances and changes in the ecosystem, you capture a lot of the 'control' that LLM-based development feels like it strips from you.

Obvious disclaimer that I use pi.dev when I am aiming for consistency, not absolute quality. Custom harnesses on pi are what I ship, claude code is still my fallback if I need to make sure a PR is the highest absolute quality

reply
My own opinion is that the "harness companies" (Anthropic, OpenAI, Cursor etc.) are fucked long-term since customers will tire of that quicksand. As the models improve, we will tire of that harness crap.
reply
> Isn't the harness basically where the frontier model companies can capture value and create a moat of sorts?

They're trying. I see a few main avenues:

1. Fitting the models to their specific harness, so that if you want peak model performance, you're stuck with their harness. But this only works if there aren't alternative models that are similar in capability or good enough that don't have that restriction.

2. Locking you out of the harness so that you eventually are just letting it do "stuff" with your data and system, although you don't get to see what the stuff is or why it's doing it. This is the pattern discussed in that Earendil blog post, "The session you cannot take with you." Encrypted reasoning tokens, secret agent prompts, perhaps eventually not even seeing what files are being read or what data is being sent back to their server. This way, you are also shackled to their harness because nothing is portable. But it only works if you trust them implicitly and don't have alternative models and harnesses that don't treat you this way.

3. Tying subscription pricing to the use of their harness, so that it's financially punitive to use another one. This is what Anthropic does. But again, it only works if there aren't alternative models and harnesses that work similarly well for you and don't have that restriction.

4. Marketing. Anthropic is leaning into this one heavy, from what I can tell, based on the constant ads I see for Claude Code. Can it work? People do overpay for things like vodka in fancy bottles that are chemically identical to the cheap stuff. Is Veblen-good AI a trillion dollar business, though?

I think the pressure from open source innovation in models and model tooling is going to make it a tough row to hoe. But I'm biased, as I'm actively rooting for Openthropic's demise.

reply
I don't think so. A basic consumer-oriented harness is a commodity (zero moat: you can ask one harness to write you another).

State-of-the-art models have more of the workflow sensibilities built in, and don't need as much help from the harness.

Where harness helps the most is very customised personal workflows (not a textbox for a prompt, but an end-to-end IDE for how you develop software) But then it's best to have your own, rather than some closed-source rigid product.

Harnesses help smaller models, and very tightly hand-holding harnesses are needed for models runnable locally. I think that is very valuable to users, but avoiding paying for the biggest models is the opposite of what the frontier labs want.

reply