deleted
replyare you working in earplugs? :)) even with 128 gigs of ram it must be super noisy.
replyI actually run it as a server - so most of the time I don't have to listen to it right next to me - it's just sitting in another room in my house - but I often am traveling with it and will have it sitting right next to my coding laptop and yea the fan runs non-stop - it's not obnoxious so i can pretty easily tune it out - also airpods/noise canceling headphones help!
replyThere’s fan noise, but the acoustic engineers at Apple have done a very good job of making it be pretty much not noticeable to someone who has noise sensitivity (me).
replythe noise is pretty doable, and so is the heat, I find. I read these messages before I had the machine and expected far worse. The tok/s is for me the dealbreaker as I prefer to have multiple sessions. For nightly runs i do like it a lot, or to be a node in a mesh.
replymy AC is noiser than my GPU server.
replyI've put Opus to it and it says it will take 3-4h to do the process to the new weights. hoping it works!
replyyou don’t need new weight. try vllm-moet from github. it will autogenerate 2-bit plane.
replyWhat kind of tps are you getting?
replyGenerally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
replyI've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.
replysame. been running it in a dgx spark and it slaps
reply