SemiAnalysis estimates their profit margin to be 70%. To be losing money on inference implies that their costs are almost 4X higher than SemiAnalysis has calculated. That's not credible.
replyI don’t see how they could credibly estimate inference costs without knowing the model size.
replyBut we do have a reasonable estimate of model size.
reply