It's endlessly fascinating to me how long it's taking for GPUs to be adopted for this kind of workload. I remember all the excitement around the GPGPU era, OpenCL, and eventually CUDA. That was like 15yr ago! Yes, GPUs can do a fantastic amount of computing. But it's really hard to make them do it efficiently. I think maybe the implicit assumption at the time was that something would come along that would make it easier. Despite continuous advances over the last decade+ it's still really hard.

I feel like we're about to learn a similar lesson with generative AI. Things don't always get easier/better/faster.