upvote

points

by anuramat5 hours ago |
comments
upvoteby singron4 hours ago|
next
[-]
Other companies were allegedly distilling the models by training on the reasoning output. By hiding the reasoning tokens, it makes it harder to do this. You can still try to distill the models, but you can't distill reasoning itself as well.

This could all be optics as well to try to give the appearance of a defensible moat. E.g. they can claim to investors that they are able to protect a significant chunk of their intellectual property this way. I'm not sure if anyone has a study about how significant the summarization is to distillation.

reply
upvoteby dragonwriter4 hours ago|
parent|
[-]
> Other companies were allegedly distilling the models by training on the reasoning output

In the case of makers of open-source models (which are also competition), there is no allegedly, they were (and still are) openly doing that.

reply
upvoteby nullc2 hours ago|
parent|
[-]
In the case of the closed models too... Claude would happily tell you it was deepseek-v3 if you asked in chinese until it caught public attention and they papered over it.
reply
upvoteby ur-whale5 hours ago|
prev|
[-]
> wdym?

https://en.wikipedia.org/wiki/Economic_moat

reply
upvoteby anuramat4 hours ago|
parent|
[-]
how is summarized CoT a moat, and how is having the top 2 LLMs not a moat?
reply
upvoteby Closi4 hours ago|
parent|
next
[-]
If you have the full outputs, it might make it easier for competitors to distil the model or reverse engineer the full process.

It may also be that misaligned responses can be in CoT which OpenAI does not want to show to users.

reply
upvoteby anuramat4 hours ago|
parent|
[-]
but "harder to reverse engineer" isn't manufacturing, that's protecting your moat
reply
upvoteby Closi3 hours ago|
parent|
[-]
What is a moat if not something used to protect the castle?

In this case it stops people copying your IP

reply
upvoteby dragonwriter4 hours ago|
parent|
prev|
[-]
Not revealing actual thinking traces prevents mdoel distillation on yhe actual output (thinking traces are a key part of the output) which makes it harder for conpetitors to catch up (a moat).

Being currently in the lead in a category is not a moat,a moat is whatever creates a barrier to competitors catching up when you are in the lead. Merely being in the lead is not a moat except in a market with strong network externalities.

reply
upvoteby anuramat2 hours ago|
parent|
[-]
unrestricted access to better models at compute prices = better synthetic data and faster research, so its not just about the product imho
reply