Thank you very much!

I think it will throttle quite soon, but I haven't tried runs longer than 30minutes with this engine.

However, there is no constant load on ssd or gpu. i/o and gpu work are alternating and there is a brief idle periods for each i/o and gpu during inference (because gpu waits for i/o and after that i/o waits for gpu)

This is where MoEs shine though. You don't need all experts in memory at once. Diffusion inference doesn't have sparse inference.