undefined

points

[-]

Right idea, but the application is incorrect.

Model training is similar to the creation of the cgi for the movie. Both happen before anyone consumes the output, and represent the up front cost for the producer.

Both a movie and a language model can cost tens or hundreds of dollars to produce.

In both cases additional infrastructure is needed for efficient usage: movie theaters or streaming platforms for movies, and data centers with the GPUs for LLMs. This is also upfront (capex) costs.

At consumption time, the movie requires some additional resources, per viewing, whether it's a movie theater or streaming. Likewise, an llm consumes some resources at inference time. These are opex. In both cases, the marginal cost for inference/consumption is quite low.

by Insimwytim17 hours ago|

parent|

[-]

  > Model training is similar to the creation of the cgi for the movie. Both happen before anyone consumes the output

I did not say anything about consumption of the output. Maybe you misread what I wrote, it is about energy consumption.

  > Both a movie and a language model can cost

But we weren't comparing cost of the movie to cost of a language model

  > can cost tens or hundreds of dollars

But we weren't talking about dollars, we were talking about energy.

We're clearly exploring different questions.

by sdenton417 hours ago|

parent|

[-]

And that energy costs money, both at the training/cgi stage and at the inference/consumption stage. It's not even an externality.

CGI renders do use a lot of electricity relative to playing back the movie for individual viewers. It's perfectly analogous.

by Insimwytim14 hours ago|

parent|

[-]

  > CGI renders do use a lot of electricity relative to playing back the movie for individual viewers. It's perfectly analogous.

I've literally laughed at loud after reading this.

I can't believe you're stretching this in a good faith.

But if you are - well, you're certainly have a unique perspective.