undefined

points

[-]

> won't stop until it achieves or exhausts your prompt

This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”.

It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons.

More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse.

Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file).

by hungryhobbit12 hours ago|

parent|

[-]

The key thing is that normally Claude grades Claude's work, so you do get a"pinky promise, as all of Claude's training bias tells it to plunge ahead (even if that means "fixing" a test by commenting it out).

With /goal, a different Claude instance evaluates the goal. This is the robust version you described, and such an "adversarial Claude's" approach, with or without/goal, is exactly how you avoid pinky promises (from one Claude).

by 9dev12 hours ago|

prev|

[-]

I'm way too afraid of a "paperclip problem"-style target overfitting to use it really; how could I even describe a goal so well it doesn't sacrifice other things that are important to me? Like, if my goal is to make an endpoint faster, will it create an over-engineered mess out of the clearly readable code I have to reach that goal?

by mh-8 hours ago|

parent|

[-]

  /goal open a draft PR that makes <endpoint> ~200ms faster while still passing tests, and adhering to nearby style+pattern

It's obviously not perfect, but it's not like you're going to YOLO it into prod without evaluating it on whatever you care about.

If you want to be able to use it like that, you need to find a way to encode (most of) your concerns into something that can be programmatically verified.

by 9dev6 hours ago|

parent|

[-]

That kind of emphasises the problem I was referring to; if I pick 200ms, will it stop before doing a proper optimisation because that would make it faster than that? Will it do something stupid to justify pushing from 195ms to >=200ms?

by mh-6 hours ago|

parent|

[-]

It might! In my prompt, I'm supposing you have some priors here about what's possible. Try saying something like "materially faster". But you need to also make sure it knows how to measure that endpoint's performance on its own, otherwise I wouldn't expect this to work well at all.

by jascha_eng14 hours ago|

prev|

[-]

Is this useful? I feel like the problem is usually not that the model isn't capable of achieving what I give it, but the way it does it. Especially if originally I didn't 100% know how I would do it myself the model often takes weird paths through the code base, takes shortcuts that end up in weird feature interactions or pulls in a dependency without weighting if it could've been done without that.

I haven't really found a good way to solve this other than:

1. Produce an initial PR fulfilling all the requirements I knew at the start

2. Chat with the model about any weird snippets I notice and talk through alternatives

3. Simplify anything that I think is overengineered or plain unncessary

Sometimes I restart all over with more precise requirements but then it sometimes makes different mistakes/takes different shortcuts.

In practice the earlier I review the better the end result imo, so /goal seems very unproductive to me?

by ubermon7 hours ago|

parent|

[-]

For some frontier models like Fable 5 it doesn't matter, but for models less trained on long horizon tasks it very useful.

by swader99914 hours ago|

parent|

prev|

[-]

It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.

by jascha_eng13 hours ago|

parent|

[-]

Can you give an example? And more curious about what you do with the resulting code afterwards I imagine its gonna be a big chunk then?

by hungryhobbit12 hours ago|

parent|

[-]

I feel like "do X untill tests are green" is sort of the prototypical /goal case.

by madeofpalk12 hours ago|

parent|

prev|

[-]

Adding a new lint rule that a lot of code violates.

by jascha_eng11 hours ago|

parent|

[-]

And that doesn't work with a simple prompt?

by mh-8 hours ago|

parent|

[-]

It's just a way of defining what "done" means, and passing off evaluation of whether "it's done" is true to a separate context and/or model. If false, it prompts the original context accordingly.