upvote
Obtaining that 64GB RAM is a meaningful obstacle for many.
reply
Not just RAM, VRAM, right? Though they're one and the same on the Mac.
reply
I'm still amazed that you can run LLMs of this quality on a machine that costs less than $3,000.

I used to assume that anything GPT-4 equivalent or higher would need $30,000+ of server-class hardware.

That said... gemma-4-12b-qat is 7.15GB on disk so should run reasonably well in 16GB, that takes it down to MacBook Air territory https://lmstudio.ai/models/google/gemma-4-12b-qat

reply