upvote
I'm genuinely curious as to what you're working on that you find 4B models good enough. I wouldn't even let a 27B model code, never mind supervising smaller models.
reply
At the moment I'm still doing shakedowns, so Typescript games compilation with a menu that has 4 games and retro artwork.

This seems to be a good example because things like the menu, high score boards etc are common, but the games are distinct. Then there's the artwork which requires decisions on look, and for coordinating.

The Qwen 4B model is multimodal so part of the AC is to view the output - I've a robust anti AI-look QA chain for that I've been using elsewhere, e.g. no floating parts, consistency, obvious missing fingers etc etc.

The longer term plan is to do some llama.cpp refactors specifically for some target hardware I have and implementing slightly different novel architectures I'd like to try (one I did already targeted CPU inference, which I did using 3 agents with specific roles; main planner, QA for planner, and benchmarking/environment handling)

The implementation was 85% of the speed of the original maxed out on my hardware but performance scaled with CPU core count whereas the original implementation plateaued. Unfortunately the break even mark seemed to be around 30 - non HT - threads.

I suppose I should look at that one again, since the increase in cores did not linearly drop off performance e.g. due to memory contention.

reply
Tell the planner to not do all that. Just plan out the high level. No code. No implementation details.
reply
Yeah, it's way better when you do design documentation, or even ticketing, to instruct it not to include any implementation specifics. You're not doing the deep dive on the zero shot that writes the ticket or the document and so it is much less informed than the agent doing the work will be.
reply
If you can run the smarter 3.8 27B model, why not use it for everything?
reply
How do you set all of that up with supervisor and minions? What harness?
reply

    > Qwen 3.8 27b is the supervisor
    > 
    > Qwen 3.5 4b are the 6-15 minions it controls
    > 
    > Gemma 4 e4b is the validator for the supervisor.

I just use Opus 5.5 and don't think about it?
reply
These are self hosted for learning experience, I could have built an agent swarm in the cloud, but I'd never have learnt the fundamentals.

- Cold starts impact, context length issues, task lifecycle management

- Inefficiencies in delegation, necessitating workflow patterns for small projects (big AIs hide this problem until you scale and they hit the same issues).

- Limits of the AI would be harder to find or notice (e.g. where time - and cost - is being spent needlessly).

reply
Or just use a more capable local model like Qwen3.8-Flash-Next?
reply
Or Kimi K3? /s

Some people are resource constrained.

reply