Architectures (like Gemma4) have things like sliding window attention that make it pretty impractical to do on a GPU, you would have to do additional things to make it work, and the talent to do that, even with LLM assistance is very small.
replyAh, that makes sense. I haven't bothered to look at current low-level NN architecture details. I'm a little surprised that there isn't a bumper crop of relatively well-informed talent to do gpgpu work. Although, when if I think about it, it has always been a little bit of a niche area.
reply