(author passing by) Could you please report the issues you've experienced on the repo? I did quite a bit of testing myself on local models (Qwen, Ornith, Gemma, Muse Glimmer) with llama.cpp, and they all worked fine. Details would be helpful.
Also one brief note for Codex users specifically. hax can work out-of-the-box as codex-cli drop-in replacement (since it can use the already authenticated codex token), so it's super easy to try it - basically it will be a zero-config experience.