> The ability to generalize isn’t just about being general purpose, it’s super robust
replyI work with LLMs daily. 5 of my specialized tasks are outperformed by a custom model than a general purpose frontier model. The performance of my custom models not only beat them but are orders of magnitude low in costs and thus are able to be used by more customers.
I suspect you and GP are talking at different layers. I think you are using robust on specific tasks with measurable confusion matrix. I think GP is talking about robust in more complex and diverse workflows, with the ability to self correct over turns.
replyEither, please correct me if I'm misinterpreting
That's part of the irony here I guess. In specialized fields, think computer vision, there were lots of teams whose innovative state of the art model was essentially just a function of the limitless compute they could throw at the problem. Now there are just people with even bigger sticks.
replyThere are lots of scenarios where specialized models still are the only option for real time, power efficiency, and so on. And transformers and other tech behind LLMs can equally produce better specialized models. But no sympathy for those who confused compute with innovation.