undefined

points

by iamflimflam11 days ago |

comments

by bilbo-b-baggins21 hours ago|

[-]

Devin shows model thinking.

I’m pretty sure the big bois don’t do it because it would undermine “confidence”.

Seeing a model output “Oh I should just delete blah. Wait blah is a production service, I shouldn’t touch that. Maybe I can gain access to blah? Oh the aws cli isn’t signed in to blah. I see kubectl has access to blah though! Wait, I should ask user permission first.”

Yeaaaaah. Thinking tokens are fuckin’ wild.

by demibabs18 hours ago|

parent|

[-]

Idk I feel like the more likely answer is to prevent distillation. Having the thinking is definitely better UX (oftentimes, I don’t know if Codex is just hanging, which it often does, or working in silence).

by geetee18 hours ago|

parent|

prev|

[-]

Running some models locally and seeing these thinking tokens was quite the experience. I never saw an LLM so "unsure" about virtually everything.

by tyre20 hours ago|

parent|

prev|

[-]

I doubt most users would look at them if they were available. More likely they don’t want to stream distillation material.

by glub14 hours ago|

parent|

[-]

I stay much more hands-on when I'm using models that display full reasoning traces. And I tend to get more things done as a result, because I know exactly when it thought of a good solution that it talked itself out of because of some invalid assumption.

Could be something very stupid like - "I don't have ffmpeg available here. Should I install it? No, I can't. I'll proceed doing something that will take me 100x more tokens and wall clock just to avoid adding a dependency." I can then just stop and say - you've got nix flake there, just add it.

That's impossible with western models. The only way is to ask why it did something stupid when it already spent 50% of your weekly quota and produced millions lines of slop.

by imron20 hours ago|

prev|

[-]

You can double click on the 'thinking' text and it will expand and you can read it. The problem is that it will often have multiple thinking/tool call sections and it can be a needle/haystack problem to find the one with the thinking you are interested in.

by tempay19 hours ago|

parent|

[-]

We don’t have access to the real reasoning text for most closed models these days, mostly due to distillation threats

by greenavocado21 hours ago|

prev|

[-]

Ah, but you CAN see the thinking if you are willing to risk your account being banned. You just have to expose a "tool" with a specially crafted definition.