Try the llama.cpp fork by thetom. It's called turboquant after the technique