upvote

points

by jacquesm3 hours ago |
comments
upvoteby mrtksn2 hours ago|
next
[-]
What models actually run? Do they generate the PWM signals/their widths to drive a servo or something? Or is it more like "joystick commands" generator?
reply
upvoteby jacquesm2 hours ago|
parent|
[-]
More like high level motion controller, so 'servo 23 1700 us' or for better grade servos 'servo 23 pos 145400', possibly concurrent with motion commands to other actuators and possibly synchronized, possibly using other kinds of protocols (G-codes).

Motion controllers are pretty impressive these days. PWM generation is too time critical to leave to high level processors, usually this is done in hardware if you want it to be guaranteed glitch free but there are also 'bit banging' servo controllers out there.

reply
upvoteby mrtksn1 hours ago|
parent|
[-]
Interesting, so how does the inference work here? are they aiming for a fixed length intervals like 10 times/second generating the positions or is it more like reactive, i.e when the request to hold a cup from a higher level comes then this is treated as a promt, the sensors are the context and when the inference finished the pulses are sent according to the output?
reply
upvoteby gopalv3 hours ago|
prev|
next
[-]
> Sensors and actuators require millisecond level responses, that's something you're not going to get out of an LLM on the other side of a communications link.

We've gotten to slightly a frame behind for the models I'm using to play video games - 36ms to get the next action.

The idea is probably not have a frontier model doing this, but to make a frontier model build these out of its own brain (well, directly edit the weights instead of RL tuning).

My team did call these sort of slices of a main brain a "Reflex agent" in similar fashion, since they do not reason only respond.

reply
upvoteby dwattttt2 hours ago|
parent|
[-]
The faster you are, the more systems you're able to control directly. At ~30ms per "choice"/output, you can choose what lower level motors/servos/systems drive towards, but you can't generate the signals that drive them. You're looking at 1kHz and maybe higher there.
reply
upvoteby baq3 hours ago|
prev|
next
[-]
> General models can run robots. They are very bad at it.

Reminder frontier labs are 6 months ahead of public models internally, if not more

reply
upvoteby jacquesm3 hours ago|
parent|
[-]
That does not make them orders of magnitude faster.
reply
upvoteby cobychapple2 minutes ago|
prev|
[-]
nothing says reflexes like getting a 429 rate limit halfway down the stairs.
reply