You're focussing on inference ... is it not more likely that A100's are being used for training/fine tuning?