I bought a trashcan Mac Pro on a whim last week ($120 in eBay!) and did some reading about them—it turns out people recently started using the GPUs to run models @ 20-30 tok/s.

I'm excited to get my mitts on it on Friday when it finally arrives.

Here's some of the resources I came across if you're interested in reading.

https://echalupa.com/blog/mac-pro-6-1-llama-cpp-firepro-d300...

https://matthewgribben.com/blog/mac-pro-6-1-llama-cpp-firepr...

What is it useful for at slow speed?