I think I've heard multiple time that a large % of training compute for SoTA models is inference to generate training tokens, this is bound to happen with RL training
replyThey can run any number of inference experiments. Like a lot of the alignment work they have going on.
replyI am not saying this would be a great use of their compute, but idle is far from the only alternative. (Unless electricity is the binding constraint?)
Electricity is charged whenever you use it or not, so very unlikely, but sure, they can find uses for it. Although they are not going to make that much money compared to claude code subscriptions.
reply> Electricity is charged whenever you use it or not, [...]
replyHuh, what? You know you can turn off unused equipment, and at least my nvidia GPU can use more or less Watts even when turned on?
Or does Anthropic have a flatline deal for electricity and cooling?