I've never understood why we can't just expose the GPU ISA directly the way the CPU does. It's all getting compiled down at the end of the day so someone has to write a compiler for it either way. We'd be substantially better off IMO if it was all built directly into LLVM and then let middleware sort out the details.
Naturally plenty of folks rather use software that doesn't take advantage of the hardware they paid for.
CPUs manage this by changing the internal micro-architecture, but historically GPUs only needed to support a graphics API and used that abstraction layer to freely change the hardware.