I've run 4, 6, and 8. Adding more video cards increases total available memory, so you can load larger models, but there are definitely some drawbacks.

More cards = more communication over PCIe. The prompts don't come in and just get magically split between each card, they move sequentially through them.

Also, with 2x24GB cards you don't really have 48GB of usable memory to load a model, closer to ~42GB + context.

And then there are power concerns, motherboard limitations (PCIe slots and lanes - a lot of motherboards with multiple 16x PCIe slots don't actually have 16x lanes to each of those slots), and more. 8x GPUs are going to easily draw 2000W on their own, if not substantially more. You'll need wiring and a circuit that can support 3000W without a risk of starting a fire in your wall.

For $5k, a single 32GB 5090 might be a better choice for a lot of people versus 4x3090s with 24GB each. It will definitely perform substantially better on smaller 27B models.

For hardware:

A good motherboard with lots of PCIe lanes (7x full 16x PCIe 4.0), DDR4 support, etc:

https://www.asus.com/us/motherboards-components/motherboards...

Add in a 3xxx series Threadripper PRO, 128 or 256GB of DDR4 (going higher becomes really expensive), and a ~1400 watt power supply. You can underpower/undervolt Nvidia cards really easily, and capping them at 250W loses you minimal performance.