Those 192GB aren't unified memory though. 128GB on Mac or 395 can be used by both CPU and GPU. It's the GPU + large memory that opens up fast local LLM inteference.
replyYeah, those aren’t. But if we had the ability to buy that much RAM in the laptop then everyone would be looking in that direction. Until this thing discussed here comes to the market, “we didn’t have computers with unified 128GB RAM either” (except of macs).
replyI'm aware you can have more. the term "SWEET SPOT" references a area that anyone/everyone can get to and isn't some magical expensive unicorn.
replyYou're doing what the IT industry has been addicted to for decades: number goes up.
> You're doing what the IT industry has been addicted to for decades: number goes up.
replyNo, I have a hands on experience with bigger models, and understand the advantages of using them.