Useful stuff, thank you. And more reading for me to do!
reply(DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).
In my experience I am getting 23-24 t/s output with dflash off, and it craters to ~9 t/s with it on, miss rate exceeding 50%. And I'm using the same device as stated on their model page/card. We might need to wait for the software to catch up
replyAs an extra data point, I have exactly the same issue on a M1 Max as well.
reply