I’ve been looking at exactly this kind of system (in a Lenovo P620) to fill with external GPUs (powered externally and ribbon cabled into the pcie slots). What would you say was your best performing model on this system? And do you get any useful work done with it or are you still dependent on SOTA models online?
replyI was able to run the gpt-oss 120b model with pretty decent performance, and the gemma models. I haven't experimented yet with qwen3 on here much. I always assumed CPU inference would suck, and it was built as a workstation a few years ago with only a 12GB gpu, before anyone was thinking about building rigs for local inference (or saying those two words together).
reply