upvote
That's the one I'm running using a fork of the gufo inference engine as gufo upstream doesn't support that quant.
reply
I'm running a regular Q4, what about that quant format justifies the fork?
reply