upvote
Yes, and if you have the PCIe lanes (say, an x16 lane - actually delivering 16 lanes! - to each GPU) it's also quite performant - it's called a tensor split in llama-server.

If your motherboard/cpu doesn't actually have those (few do outside some xeons, epycs and threadrippers) you can still do it - it's called a layer split and will work even with 1 lane per GPU. Each GPU will work at its maximum speed, but only 1 will be active at any given instant - imagine a relay race.

(Didn't mention which PCIe generation - obviously the higher the better. At v4 and up, even 8 lanes per GPU would be enough for a performant tensor 4-way split)

Edit: If you have more than 1 user at a time, the GPU can actually all be working all the time, if there are enough parallel requests to serve. But you need enough KV cache for all the sessions you're running in parallel.

reply
I do use 2 amd gpus and I get high 40 for generation, 500 for pp and low 20/100 by the end of the context of 256k.

llama-server --host 0.0.0.0 --port 8089 -m Qwen3.8-27B-UD-Q8_u.gguf --spec-type draft-mtp,ngram-mod --spec-draft-n-max 3 --spec-draft-n-min 1

if you have an igpu and want to exclude or just use some gpus you can use

--device Vulkan3,Vulkan2,Vulkan1

in my case vulkan because of amd, you can see your devices with

llama-server2 --list-devices

Available devices: Vulkan0: AMD Radeon Graphics (RADV RAPHAEL_MENDOCINO) (33515 MiB, 29349 MiB free) Vulkan1: AMD Radeon RX 7900 XTX (RADV NAVI31) (24560 MiB, 4911 MiB free) Vulkan2: AMD Radeon RX 7900 XTX (RADV NAVI31) (24560 MiB, 7681 MiB free)

reply
Yes, you can. Ideally though, you want to minimize the number of cards and maximize the amount of memory in each card.

Using multiple cards is one of the things that the models and software that Unsloth releases does really well in terms of ease of use and relatively good performance.

reply
How many do you recommend?
reply
I've run 4, 6, and 8. Adding more video cards increases total available memory, so you can load larger models, but there are definitely some drawbacks.

More cards = more communication over PCIe. The prompts don't come in and just get magically split between each card, they move sequentially through them.

Also, with 2x24GB cards you don't really have 48GB of usable memory to load a model, closer to ~42GB + context.

And then there are power concerns, motherboard limitations (PCIe slots and lanes - a lot of motherboards with multiple 16x PCIe slots don't actually have 16x lanes to each of those slots), and more. 8x GPUs are going to easily draw 2000W on their own, if not substantially more. You'll need wiring and a circuit that can support 3000W without a risk of starting a fire in your wall.

For $5k, a single 32GB 5090 might be a better choice for a lot of people versus 4x3090s with 24GB each. It will definitely perform substantially better on smaller 27B models.

For hardware:

A good motherboard with lots of PCIe lanes (7x full 16x PCIe 4.0), DDR4 support, etc:

https://www.asus.com/us/motherboards-components/motherboards...

Add in a 3xxx series Threadripper PRO, 128 or 256GB of DDR4 (going higher becomes really expensive), and a ~1400 watt power supply. You can underpower/undervolt Nvidia cards really easily, and capping them at 250W loses you minimal performance.

reply
I am getting 14 t/s on my 16 GB card at full context with the UD-Q3_K_XL quant. Model link: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF.
reply
For some reason the unsloth models leave hardly any room for context. I've switched to the regular (non-unsloth) and get about 25 t/s and get about 80,000 more context tokens for the same quant.
reply
Wow, interesting. What KV cache quantization do you use?
reply
Yes, google "llama-server split model multiple GPU" for some concrete examples.
reply
Of course. Models don't actually require VRAM. Nor do they require regular RAM. You could have 1 GB of RAM and swap the model to disk as you need different parts of it. And if you didn't have enough disks you could access weights via a network connection.
reply
You don’t even need electricity. You could print the model weights onto millions of sheets of paper, and hire a team of carrier pigeons to fly them into your office one by one. No VRAM!
reply
Coincidentally that’s also Google’s new method for distributing Android sources.
reply
The bitrot would be excessive
reply
The key to running in lower amounts of VRAM is patience. It'll be slow, but it'll work.
reply
Yes, but unless they support NVLink (they don't), it's quite slow.
reply