That's only bringing up the horizon maybe 1 year. At industrial scale - I guess something like an Amazon warehouse could start to be interesting with a local datacenter IN the warehouse within 2-3 years.
replyFrom a cost perspective, I just can't imagine that heavily, heavily specialized models wouldn't crush a general model, though - which Amazon already does.
Everyone thinks specialized models are the ticket, except then they aren't: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
replyI wonder to what degree that holds up for very limited domains. You have to balance model size, power budget and cost which is a completely different ballgame than looking at a stationary server with a near infinite power budget and where the costs can be amortized over much larger numbers of hits.
replyRobotics tends to be at the worst possible intersection point of all three of those: limited power budget, relatively small model size and cheap enough to be affordable when embedded into an appliance.
Kitchen appliances run Linux now. I think the economies of scale might win regardless
replyTraining specialized models for every task is a "general method leveraging computation" in the sense of the Bitter Lesson. Making the best possible model for each task is guaranteed to be at least as good as making one model and using it for everything, and often it'll be better.
reply