Do you think we are near the theoretical maximums of how much model we cram into 20B?
replyI don't consider myself qualified to make that call, though I believe there's plenty of room for improvement. But I don't mean inference compute, which is fairly accessible for well-off enthusiasts even now. Training is a lot more expensive though.
replyBack in the day we had seti@home, prime95 and fold@home. We will make it work.
reply