This comment has me a bit confused.
replyConsumers were complaining about the standard 8GB with the early 2020 refresh of MacBook Pros, many OSes ago. Sure, it might be workable for many tasks (as evidenced by the recent sales of the MacBook Neo), but users with a mere 8GB shouldn't have expectations of LLM performance. Even 16GB feels like a stretch.
I think you are mixing up RAM and VRAM.
replyOn a Mac they are the same thing; they're shared. Of course you need some amount for the OS, but if you have an Apple Silicon Mac with 24GB of RAM, you can likely run a 16GB model.
replyThey are effectively one and the same on Apple Silicon.
replyWhich most people as a matter of fact don't use. A majority of people with laptop have separate memory pools and the VRAM of them is nowhere near that and even on most gaming laptops you aren't getting 16GB VRAM.
reply> A majority of people with laptop have separate memory pools
replyMajority of people with laptop have RAM and igpu using some of that as VRAM.
I would say on this forum it wouldn’t be suprising for commentors to be near or above 50% that have access to an M Series Mac…
replyUnified Memory or VRAM, not just RAM.
replyThey already provide E2B and E4B that run on (much) smaller devices, including tablets and phones. This fills the gap in the middle. The bigger Gemma 4 models are excellent for their size, but at 8-bit quantization they need about 64GB of VRAM or unified memory. 48GB for 6-bit. Any lower quantization than that, they start to get notably dumber. So, a 12B is interesting for that middle ground.
replyI have 24 gb unified memory so it’s a good model for me
replySurely they must know the current hurdles, but clearly they know that all the relevant people are monitoring the market for the proper hardware to get and 16GB will be an entry point.
replydeleted
replydeleted
reply