Depending on what you are doing you may find all the models are written for CUDA though. I'm playing with gaussian splatting and it's like that. Codex is doing a good job porting stuff to Metal, but it would be easier & faster if I had Nvidia hardware to just run the models natively.

Not everything with AI is LLMs running on llama.cpp.

That's the thing, my understanding is that these Nvidia ones are supposed to have very impressive performance.