We're allowed to have our ceremonies.
replyThank you. If this whole thing isn't fun, it isn't worth doing.
replyYou don't think it's useful to learn whether a model's "intelligence" generalizes beyond the tasks and modalities it is usually optimized for?
replyAbsolutely not. Makes about as much sense as judging a car based on how good an airplane it makes.
replyHave you considered that the single most impressive breakthrough of LLMs as a technology is their ability to generalize beyond what they were explicitly trained on? Great analogy, pal, but LLMs aren't cars.
replyI disagree. If GPT-7 can draw the Mona Lisa in MS Paint via computer use, this would be interesting.
replyThat it isn't the most efficient way to achieve the same end result is irrelevant.