undefined

points

[-]

Unfortunately in xhigh thinking it goes down rabbit holes in such an extreme depth-first way, that whenever you choose to cut it off, there is a very good chance it will not have got round to musing on even half of the prompt! It doesn’t really obviously loop in xhigh, so I am not sure if an “overthinking guard” proxy would have much to go on, but it does obsessively ruminate on edge cases. I have seen it overcomplicate simple code as a result even in my limited testing.

Probably the better solution if you want it to be quicker but still fairly thorough appears to be to configure reasoning effort instead of thinking budget. It seems to do very well still even on the Low setting; on the Medium setting it can get stuck in loops like 3.6 does.

I think xhigh reasoning effort was an absurd choice for a default, and so was not sorting out the chat template so LM Studio could offer the reasoning effort dropdown.

by cyanydeez1 hours ago|

parent|

[-]

to the point though: most of that overthinking is useless if you have a proper redirect message. So setting arbitrary budget and getting it a good message will do the trick regardless of what type of thinking it's doing. The reason thinking seems to work is that it's just trying to find an optimum outside the local optimum, and the thinking trace helps find it.

The only think I could think that'd be better than the --reasoning-budget would bet a budget jitter just in case it really is repeating a pattern and you want to escape it arbitrarily, otherwise yes, it could keep looping if you're always cutting at the wrong time.

by dofm1 hours ago|

parent|

[-]

> The reason thinking seems to work is that it's just trying to find an optimum outside the local optimum, and the thinking trace helps find it.

Yes, I think I finally have an intuitive sense for that. But surely on a longer prompt it is still better for the final response if the thinking has at least brushed past all of the prompt?

One of the things I witnessed with xhigh is that while the thinking trace starts out intending an overview of the prompt, it actually can go fully down a rabbit hole off one of the first two or three bullet points even when it was seemingly intending not to.

It’s basically a lot like me. Gets sidetracked by the interesting bits.

by bitexploder2 hours ago|

prev|

[-]

Yeah, but be fair. Working with small models is a different ball game. Not all the batteries come included :)

by bellowsgulch1 hours ago|

prev|

[-]

This is definitely such a cool feature that I wish cloud providers would expose.