Efficiency is an angle of research that's quite active, see quantization, MoE, speculative decoding, QKV cache optimizations, sparse attention, etc