How does that compare with the bf16 version?
replyFor my "Please recite Jabberwocky" test the bf16 almost passes but the ternary and even fp8 versions fail badly.
M1 Pro, same prompt, same cli options:
reply 32,706 tokens
38min 19s
14.22 t/sdeleted
replyHonestly looks pretty good except whatever is going on with its booty. Is that an ass helmet? I cannot parse what's going on there.
replyLike you've never worn an ass helmet
replyOnly because I hadn't previously thought of it xD Step up from the standard ass-hat for sure.
replyI think it’s supposed to be a wing
replyI like the lens effect behind the rear tire.
reply