Yes, you are right, and I was too optimistic. I'd have to level up.
replyThis is the stuff I've put together and it is clearly too small for this:
https://jacquesmattheij.com/local-inference-with-large-langu...
Could you mention the exact model of the PCIE switch/host adapter you are using? The Aliexpress links are always a bit ambiguous. I am working with the c-payne switch https://c-payne.com/products/pcie-gen4-switch-5x-x16-microch...
replyThe system is a $25k custom build with 8 video cards (4x RTX PRO 4000, 2x RTX 4090, 2x RTX 3090). That price was only valid for early 2026.
reply