Perhaps I should just google it, but I'm under the impression that ollama uses llama.cpp internally, not the other way around.
replyThanks for that data point I should experiment with ROCm
I meant ollama uses llama.cpp internally. Sorry for the confusion.
replyFrom what I understand, ROCm is a lot buggier and has some performance regressions on a lot of GPUs in the 7.x series. Vulkan performance for LLMs is apparently not far behind ROCm and is far more stable and predictable at this time.
reply