Agree, I remember when even half precision made its way into C# sometime around 2020 (I didn’t know much about ML then) and I thought, well I guess that’s a worthwhile tradeoff but I can’t imagine going lower. Lo and behold (1-bit Bonsai) how much lower you could go.
replyTernary?
replyKnuth's base-e proposal enters the chat.
replyThey were right about everything 50+ years ago, but they didn't have the budget for the right hardware, had to write conference papers and books instead.
I can totally see how ternary would work from a physical implementation perspective but I have a really hard time visualizing anything using base-e, can you explain how such a thing would work in practice?
replyNo it's impossible. But it would be optimal!
replyAh, the spherical cow of number bases :) Thanks for the response, that saved me a sleepless night.
reply