Show HN: Shoehorn – Quantize any model down to run on your machine
(notactuallytreyanastasio.github.io)
The project name is perfect!
replyI gotta laugh at some of the models it suggests, for example:
reply> AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF
you’re telling me you managed to fit Fable 5 into just 4B?
I gotta laugh at your thought process: knowing Fable 5 is a large frontier model, you're telling me that the first thing that came to your mind on seeing that model name is that it's a quantized version of Fable? As opposed to a distillation/fine-tuning on Fable responses?
replyWell to be fair here... the title of this post doesn't mention fine tuning, it mentions quantization.
replyDon't make fun of people you think are ignorant, it's a pretty shitty look
replyWell, then don't get all snarky and dismissive of things you might not be knowledgeable about ("you" here referring to OP).
replyReminds me of https://github.com/AlexsJones/llmfit
replyLLMFit tells you what can run on something. I built something quite similar to their search into Shoehorn now.
replyThis is interesting. I wonder how it could work with something like https://github.com/JustVugg/colibri.
replydoes this work similar to airllm? i am wondering how it would handle something like quantizing kimi k3 on a budget of 8 gbs, or is that something you are not attempting to solve yet?
replyYes that is exactly what this does.
replyCould you explain what happens when you try to shoehorn a 2.4T parameter model into a 24gb m4 mac?
replyextreme divergence would be my guess
replyWondering the same thing but for 48gb M5 Max.
replytried it out but based on the model sizing result i got i got an insufficient memory error when the server started running
replyIf you could post an issue if you still have the error around that would be awesome.
reply[dead]
reply