Thanks for sharing this - I would not have looked into it otherwise. Apparently model retention is ~98% of the original at IQ1_M if I read that right. Crazy. Excited to try this now.
replyQwen3.8-Flash-Next at Q2 is often looping itself with repetitions.
replyYou don’t know what you don’t know
replyIt works the other way too! People don't know that 3 or 2 bits can still be good because they've read the hivemind's opinion that <4bit is unusable trash and they've never experimented themselves. The reality is a lot more nuanced than that, especially for models as large as 284B.
reply