(github.com)
The underlying stack definitely can connect to multiple robots simultaneously. Our current implementation is sequential, where the language model can connect from one robot to the next (and then back).
But it is definitely possible for us to write it to be simultaneous as well.
On the flip side, how would you handle conflicting commands from multiple clients? Is it last-writer-wins, or do you envision some arbitration layer? It feels like orchestration + conflict resolution will be key if MCP is to scale beyond single-robot demos into fleet-level use.
We're also building out a technical steering committee to help guide our direction on topics like this. Safety is a big category where having direction from across the community will be important.
We're using websockets as the interface between the server and the robot itself, which to the best of my exploration does have the ability for simultaneous connections.
Does this also entail industrial (sector-agnostic) applications where mitigating actions, based on vision or other sensor data based leading indicators, can proactively be taken using LLM-directed mitigation protocols? Does it allow for non-technical users to perhaps drive debugging or other similar mitigation actions?
For more advanced use cases, we’re also thinking about adding validation layers and safety constraints before execution — so the MCP acts not just as a bridge, but also as a safeguard.
This is the video of interacting with and debugging an industrial robot. (A few of the other comments here have been talking about this, that we see some amount of what looks like emergent behavior) https://www.youtube.com/watch?v=SrHzC5InJDA
This is a video from a collaborator research lab controlling a Unitree Go (robot dog) https://youtu.be/RW9_FgfxWzs?si=o7tIHs5eChEy9glI
What excites me most is the potential for MCP to help with diagnostics and deployment for non-developers. A lot of lab techs or operators don’t want to dive into ros2 topic hz or parse logs — they just want to ask simple questions like “why isn’t the arm responding?” or “is this topic publishing?”.
A natural language layer over ROS could make debugging and deployment way easier for non-technical users — almost like having a conversational ros2 doctor or ros2 launch.
This isn’t just a bridge between LLMs and robots, it can also be a bridge between non-developer operators and the ROS ecosystem.
For the industrial robot (in the video on the main readme page) I intentionally gave Claude no context beforehand. All of the inferences that you see there are based on information that it got through the MCP tools that let it analyze the ROS topics and services on the robot.
In fact, I had a starting prompt to ignore all context from previous conversations because this looked to be like an example of emergent behavior and I wanted to confirm that it was not picking things from my earlier conversations!
The demo video with the Unitree Go (robot dog) uses this approach to give the LLM additional context of the custom poses available to it.
I sometimes face issues with LLMs running out of tokens or only using partial contacts from previous conversations - thereby repeating or compounding on previous incorrect responses.
Any thoughts on how to tackle that? Or is that too abstract a problem/beyond the scope to address at the moment?
So far, I've observed that for Claude and Gemini, which are what we've been testing most with, the Language model has been pretty good at recognizing its a faulty initial interpretation when it queries more information from the system.
Running out of tokens is a more significant issue. We saw it a lot when we queried imaged topics, which led us to try writing better image interpreters within the MCP server itself (credit to my collaborators at the Hanyang university in Korea) to defend the context window. Free tiers of the language models also run out of tokens quite quickly.
PS - Thank you for the questions, I'm enjoying talking about this here on HN with people who look at it critically and challenge us!