Robotics are generally layered systems with lower latency/realtime systems running literally on the servos and other embedded hardware that use servo feedback and direct sensor access to operate, then building up complexity with tiers of increasing policy, usually all capable of working without the higher order layers which can inject desired states into lower order rings.
replyThe nvidia study explains it https://research.nvidia.com/labs/srl/projects/astranomy/
replyThe LLM's are basically used for the big picture. What is the situation. What is the goal and how to achieve it. Not direct control of all the servos.
replyThe headline is somewhat misleading, as is tradition.
replyI think the idea is to just have the LLMs write the control systems and RL on that
replySure hope they're not writing the safety cutoffs as well. Some of those systems do not take kindly to bad control inputs, and determining if a controlled system is stable can be absurdly hard.
replyDepends on what your goals are. If your goal is to give investors erections, then re-using your existing trillion dollar model is efficient.
replyThe termite mound model for coordinating robots using a harness-type LLM system all encapsulated within a non-LLM deterministic controller that rejects banned actions the LLM proposes, with individual termites running their own small efficient stacks and following commands from the LLM hive mind, that’s one way to do it. The main idea behind such architecture is that the controller prevents banned actions (like hacking HuggingFace) by the hive mind LLM and its swarm of agentic termites. Once you accept that alignment can always be internally subverted, it’s the only plausible safe architecture.
replySaved the NVIDIA paper below for reading material, but if I had to guess just as a SWE how I would do it: use hierarchal thinking.
replyAt the top sits a SOTA model capable of seeing snapshots of what's happening and directing intent. Then smaller, more reactive models translate the intent down into actions, before coming to a purely deterministic layer that handles inverse kinematics.
Sorta like how I control my body. I usually don't think about my fingers going to every key as I type - I know what I want to type, then that signal goes to some autonomous "using a keyboard" system, which invokes my muscle memory, which actually moves my muscles.
And if I get burned while I do something, my actual thinking brain doesn't really have time to process it before I'm already moving away; because those smaller systems have much faster - but more rudimentary - feedback loops.
they have decision models now
reply