It's also shockingly twitter-nerd-coded. "The cure for token anxiety", it advertises. To be honest it's hard to see why anyone would buy this product so maybe they decided to take a wild swing with the marketing. The only use is people who really, really, want to run models locally vs getting a much cheaper and higher performance result from a cloud host.
replyHigher performance, I agree, but cheaper?
replyI didn’t check how much this costs, but if you use AI locally a lot, it’s going to be amortised pretty quickly. Burning 100$ a month on tokens has become insanely easy. I remember when it was unimaginable for me…
You top out at 20 tokens per second on hardware with memory bandwidth this low for any local model actually worth using. Doing the maths, it’s not financially worth it. Only worth it for privacy and control reasons.
replyI do love my GB10 Asus Spark-like though still!
I don’t understand your calculation, can you elaborate? At 25USD/Mtk output, assuming your 20tk/s, I generated/saved (minus power costs) ~15k$ in a year.
replyGranted, it won’t run 24/7, but over a couple of years, this is definitely cheaper.
Speculation is it’ll cost at least $6k for 128GB
reply