Electricity cost seems to be about $30/month for a 32B model on a GPU. It's probably better on Apple hardware.
replyhttps://github.com/QuantiusBenignus/Zshelf/discussions/2
Not accounting for hardware, of course :)
My Mac Studio uses about 60–80 watts whenever I’m running a model (as measured by the system metrics), so it’s less than 2 kWh/day at full blast. Electricity is like 0.125 €/kWh, so that 24-hour period would be <0.25 €.
replyNot accounting hardware in my costs, since I didn’t buy my hardware for running models. Running models is just something it can do in addition to what I got it for.
The price, processed tokens, and output can be anything, it just depends on what GPU it is.
replyNvidia GPUs are much more efficient than Apple hardware for inference(and training).
Cool beans. You're not the target audience then.
replyDid I claim I was? I just said why I and people like me are not talking about it.
replyand he said its cool
reply