upvote
Any one weight, but all of them. And also crushing the architecture itself?

I wouldn't pick up 400gb of hardware to run in that mode. I might try it for fun, but even then you are looking at handling a 95GB active parameter set.

This is NOT a model for most home labs. I'm sure some can and will use it. But most, should steer clear.

reply
KL divergence (you misspelled it) doesn't tell you anything about capability drop - how much did this particular benchmark (thus ranking among models) change after 10% or 50% KL divergence?
reply