upvote
96GB is what the author claims will work in a foolproof way for easy production use. But nothing stops you from trying to run it on 48GB, it ought to gracefully fall back on accessing model layers from the disk.
reply