I feel like good engineering doesn't just ignore those things, or at least it didn't before recently. Now I guess social media has added a pressure to reduce everything to a hot take.
I can't see any benefits that a typical ML classifier would not be better at.
This probably just means that I could have been reaching for that tool more often already. But in practice I wasn't, and this has opened my eyes to the potential opportunities there.
One advantage of using generalist models is that the generalists are improving - regardless of whether you're doing anything about it.
Being able to route prompt to features that then route to special models would be a really solid implementation.
Agreed. This isn't new. I led a research team at a Fortune 500 that used a transformer based classifier approach in a commercial product as far back as 2022 and we didn't come up with it. It was already common enough that we found the inspiration for our implementation on some web forum. Models like RouteLLM have been around for a long time. The news here isn't that a new model type came about, its that a large percentage of people messing around with this stuff that are new to AI just learned that not all transformer based implementations need to be autoregressive.
It doesn't have to be new, it just has to be consumable by devs.
You could send text before Twilio. You could process credit cards before Stripe.
Jev, at the end of the day is an easy to use API.
Everyone seems to forget that usability is a thing.
I think building generalist classifier is some open ended research task, where frontier labs can contribute: different internal reasoning, instruction tuning, building datasets and benchmarks, building and distilling super large models.
and then whatever tech it is will be absorbed/assimilated/Sherlocked into the leading products anyway