> The joke is that macs are famously slow at prompt prefill and you are not getting anything back in 3 seconds.
replyYour knowledge is out of date. In truth it depends on the Mac and the models used.
I asked this question on M5 Max 128GB, using Ollama model Quen3.8:27b-mlx, with thinking enabled.
Question: "Give me a python code snippet that opens a file and sorts the lines of text. "
In 2.4 seconds it gave me 4 examples that work with different sorting configurations and a summary of when to use each.
Compare that to an older model of gpt-oss:20b, took 5 seconds to finish thinking and 2 seconds to stream the answer. It gave me one python example snippet and two one liners that do the same thing.
We are talking about models of the flash size, 100s of billions of parameters, don't listen to the media, size does matter
replyI was just pointing out your claim that you can't get a response in 3 seconds. If I had asked the model just for the code it was under a second.
replyLocal models are good enough that it's not an issue.
But keep changing the goalposts if it makes you happy.