upvote
Energy costs (and almost everything else) are a minimal part of providing inference.

Most of the cost is depreciation on the GPUs and the HBM.

The one number you need to figure out is how much it costs you to buy that GPU and HBM, then how many tokens you can sell from it over its lifespan which is generally amortised over 2 years.

reply
This is not correct. Their margins are extremely high, (~1000%), and none of the factors you listed would cause them to have negative margins.
reply