upvote

points

by badsectoracula6 hours ago |
comments
upvoteby Vaushite5 hours ago|
next
[-]
> There isn't any real harness driving this, i just give it a task, tell it how it'll be handled and off it goes.

The tool that implements this is the harness. An LLM by itself cannot "go off" and "do things" - it can only complete text blocks. A harness bridges this to real world actions or just a chat box interface.

reply
upvoteby badsectoracula2 hours ago|
parent|
[-]
I can't edit it to clarify, but what i meant is that the process i use isn't driven by a harness but done "by hand", not that there isn't any harness at all. The PDF describes a harness that is actually used to drive the manager/worker role.
reply
upvoteby BubbleRings4 hours ago|
prev|
next
[-]
This is close to how I do it also. Except I have never interrupted it in a task, I always worried it would destabilize it and it would leave some function half finished or whatever. I just have standing instructions for it to pause places and ask “ready for the next phase?” Do you find it responds well and correctly to being interrupted in the middle of things?
reply
upvoteby badsectoracula2 hours ago|
parent|
[-]
The model itself (Qwen 3.8 27B specifically) can handle being interrupted just fine and i often interrupt it to clarify things when i see it finding my instructions vague or to add some extra info.

However there is a bug somewhere between my instructions and the model - i think it is the jinja template, though i'm not sure as i haven't actually looked too deep into it - that in some cases make my last message be sent repeatedly to the model after each tool call, making the model think that i'm getting impatient or something :-P. When i notice it, i tell the model that this is some harness bug and it can ignore the repeated messages, which it does.

It seems to happen when i stop it while it is thinking after a tool call but i'm not certain. I try to stop it while some tool call is in process so it cancels the call as that seems to avoid the issue. Restarting llama-server also seems to make the issue go away so it might be a llama.cpp bug, but TBH i haven't found a clear repro case aside from it happening when i interrupt it while it is thinking. I think it is more common when the context is close to full, or at least i mainly remember it happening when i've been using a session for some time (i.e. not in fresh sessions, which does make reproducing it a bit harder).

In any case, the model itself seems to deal with all the above fine (and accepts the bug explanation for repeated messages), it is the rest of the stack that can show an issue with interrupting it (but even that is relatively rare).

reply
upvoteby BubbleRings2 hours ago|
parent|
[-]
Thank you! Yeah I think I would have experimented more with interrupting it by now, but my jobs only recently got longer and multi step. Plus I could never figure out, if I do want to interrupt this, what’s the best time to do it?
reply
upvoteby 38362936483 hours ago|
prev|
next
[-]
No, pi is the harness. You may have chosen the most minimal harness available, but it's still a harness.
reply
upvoteby badsectoracula2 hours ago|
parent|
[-]
As i wrote in the other comment, i know, i just meant is that the process i use isn't driven by a harness but done manually. The PDF i quoted mentions a harness does that driving so i wrote "There isn't any real harness driving this" to mean that i do that by hand instead of being done automatically.
reply
upvoteby 4 hours ago|
prev|
next
[-]
deleted
reply
upvoteby verdverm6 hours ago|
prev|
[-]
This is pretty close to a process I use (regardless of model size) and I would agree getting it into a workflow I can run with a one-liner is a good idea. There are a few ways of doing this and I have tended to fallback to markdown engineering. (custom agents and subs in opencode)

It's really interesting to see the progress in the 27-30B range, they have gotten quite good but still make noticeably more hallucinations or bad assumptions (ime).

reply
upvoteby octoberfranklin5 hours ago|
parent|
[-]
Yeah unfortunately the confabulations are the real problem with models of this size.

IMHO you need at least 500B weights before the confabulation problem is really under control. i.e. DeepSeek 4.1 or GLM.

I admit some days I'm just lazy and throw a whole Kimi or QwenMax at it.

reply
upvoteby electroglyph2 hours ago|
parent|
next
[-]
size doesn't really have a correlation if u look at the hallucination benchmarks. this is mostly mitigated thru post training imo. also, deepseek 4.1 is notorious for hallucinations, so kinda ironic to mention it as an example here =)
reply
upvoteby verdverm5 hours ago|
parent|
prev|
[-]
> before the confabulation problem is really under control

It's definitely much better at bigger sizes, but I've still seen it at the frontier when coworkers ask me about changes their Ai made to code I originally wrote. (a good habit that organically emerged on the dev team)

reply