My assumption is the hardware will have one/several smaller models running on it for locomotion, balance, etc.
Then the hardware "phones home" for decision making and planning.
Essentially the planning doesn't need to be close to the device as long as the local models can handle some part of the plan execution and make sure the robot doesn't fall over. The latency shouldn't be an issue. In fact more powerful centralized compute quickly creates an advantage over slower local compute in terms of end to end planning.
Sooner or later, the models will stop improving. After that happens, the best models will be etched into model-specific ASICs with weights fixed in mask ROM (like Taalas is already doing), which will result in massive, at least order of magnitude inference efficiency increase.
And although the industry hit a wall in density/area, they switched to scaling in 3D or glueing chiplets together. Moore's Law is still working if you change the definition to transistors/integrated circuit.
Also, your model probably doesn't need like a lot of its knowledge to control a robot.
From a cost perspective, I just can't imagine that heavily, heavily specialized models wouldn't crush a general model, though - which Amazon already does.
Robotics tends to be at the worst possible intersection point of all three of those: limited power budget, relatively small model size and cheap enough to be affordable when embedded into an appliance.
Robots stand-alone AI do not really benefit from things like batch mode so you want to keep your LLM as high level as possible and your robot's subsystems as autonomous as possible, very light coupling between the two, definitely nothing time-critical. And that's the challenge: in robotics almost everything is time critical.
It just seems like jumping the shark a little bit.
It's basically completely unviable for a few years still.
I don't think it's viable in the sense that they are still in a demo stage.
Also, I would assume these models are going to get bigger before they get smaller (but they will get smaller over the long run)
For a set of tasks that almost no one would use a robot for...