upvote
And is this using conventional model loading (all in VRAM), or are you streaming it in some way?
reply