ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point was just that training an LLM takes more resources and expertise than distilling from an existing LLM so I don't think the equivalence between training and distilling is entirely justified.
It's the most CS-major take ever!
Or maybe they're going through an intermediary "transfer station" that's breaking terms of service:
https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...
Anthropic's own copyright infringement could apparently be forgiven for 1.5B USD after all, so maybe there's a price that breaking the distillation clause for is acceptable too. Or some other arrangement.
There is no world in which me vacuuming the entirety of human knowledge to make a genai model is ok but hoovering my model answers is not. The hypocrisy is stunning and risible.
Now if you go and make a model based on purely synthetic data and not a single work made by humans, you would have a valid point.
Like look, I'm not a native speaker, sure. But I think when someone says "value add", that means there was value there (which you claim they're rhetorically erasing), and then that was added to. Under no interpretation of this phrase do I get an erasure of prior value.
So certainly, as long as words mean anything, no, they absolutely did not say or suggest what you claim they did, and what you extract a thus unreasonable amount of obnoxious schadenfreude from, while throwing in an insult for funsies at the end.
It's the second time I feel compelled to reach for this just today: https://i.kym-cdn.com/photos/images/original/002/659/979/108...
The LLM output, is not the same as the input - there is value add.
Of course works used as raw inputs to LLMs required work and are reasonably subject to IP concerns - but they are different.
It's possible that the LLM makers 'owe' the content creators that created the content they used to make their products - it's an interesting but separate question.
We could very well end up where content IP is protected, LLM output is not and visa versa with reasonable legal founding, doubtful but plausible.
But it's debatable if that's the case.
Google stores copyrighted content and produces in in their product.
Also - it's fair game to use snippets of things here and there, if the derived work is novel, which I think it is for LLMs, mostly.
I do agree though, that we ought to draw the line somehow.
How, and why?
> We could very well end up where content IP is protected, LLM output is not and visa versa with reasonable legal founding, doubtful but plausible.
That is the current state of legal rulings - LLM output is public domain, not copyrightable.
Our current laws simply weren’t built for this and I expect the legal status of LLM output is not going to be resolved until Congress actually legislates on this topic.
How, and why?"
How are they even remotely the same?
They're not even used the same way.
One is raw data input, the other is training content - designed to train LLMs.
One is a set of IP derived for other purposes entirely, and has esablished IP law - how you can use someone else's creative work or not ... for LLM outputs, less clear.
Writing books, building Wikipedia, and answering questions on online forums takes a lot of resources and expertise that scraping didn't. So at the very least, we're already one rung down the "maybe you should've asked" ladder.
This is not automatically true. Training and distillation use the same underlying infra and method and there is no intrinsic differences in between.
It took me a year to write a book. It took OpenAI and Anthropic a fraction of a second to ingest it. Do you understand now why I give zero shits if it takes Anthropic a billion to train a model, and Moonshot 10k in API cost to distill it?
If that is the whole point you need to clarify why this is the case on an objective level.
I would say building a comparable model using any means necessary (just like what Anthropic and OAI did) at a lower cost is actually more valuable to soceity and Monshoot is arguably generating more value with less.
If the distilled model is cheaper, then it's just LLM's getting LLM'ed.
Raw materials vs. Value add.
They are different things, like ore and metal.
Distillation is a new thing we need to understand, it's probably closer to IP than not.
We're talking about things like text people wrote, not some kind of raw data floating out in the ether.
Ore has value, a different kind of value than the output of the refinery.
https://arxiv.org/abs/1503.02531
although i doubt there has been a legal case over it yet in the context of the legality of stealing shit but IANAL.
It's completey insane that we still don't know how Open Source would work, that the laws are vague and we're still technically waiting for the courts to decide on cases.
The government should a) legislate and b) create test cases and run them through the courts so that we can have clarity.
That would stun me, but it's a little hard to read.
The recent announcement that AI-assisted research produced a counterexample to the Jacobian conjecture--a long-standing open problem in algebraic geometry--shows the original value AI can create. The result was not copied from a textbook; it emerged from AI learning from existing material, much as a human does, and then applying that knowledge in a new way. If that's a violation of copyright, then a human doing the exact same thing would be a copyright violation too. But it isn't.
True, if the human's access to the book was legal
A great deal of training was on the open web, no one should complain.
But at least Meta and Anthropic were caught red handed taking copyrighted works, illegally, for training
I think international IP laws are too strick and onerous, but they were broken to train these models
It’s massive copyright infringement.
The human buys the books.
I wouldn't want to live in a world where technology or general people's wellbeing was held back by obsolete laws that ended up lingering on just to protect undeserving special people at the expense of the rest of society. Remember guilds for tradesmen? They were also a monopoly given by the government to special people. They had their purpose but nowadays we have different ways to keep tradesmen working effectively like license requirements and insurance.
Just to be clear, I think we do still need copyright, but that we might be in a transition period where it has to be redesigned to adapt to AI.
> protect our first-party products from abuse like bots, scraping
Won't you think of the trillion dollar corporations?!
>Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.
If the distillation is irrelevant to why it is competitive, why do they do it then? Obviously is helps improve their benchmarks/performance to some degree, otherwise they wouldn't need to do it.
Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively.
If Kimi k3 really were above Fable 5 then there invariably the USG would have to consider their restrictions on model capabilities excessive, or one would have to admin closed source models are held to more restrictive safety standards than open source models.
>Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively.
How would you know this? How could you ascertain exactly how much performance is attributable to their unique engineering/research? If they really were so competitive they could surely make a model that isn't dependent on distilling Fable or other frontier models.
Kimi specifically relies heavily on reasoning traces which is largely due to their training strategy and will perform poorly when thrown into a conversation from another model. Another fun advancement is that they simply ctrl+c ctrl+v'd attention which means that the model can steer where to look in the context window without ever producing an output token increasing token efficiency and attention accuracy as a side effect you end up with weaker prompt adherence.
None of these 'issues' manifest in US models which proves that kimi has diverged and is achieving these capabilities seperately from the architecture that US labs rely on.
I would agree with you during the Deepseek R1 era, but US labs were heavily inspired by open research at that point as well so I wouldn't give them too much credit.
In other words: even if the US (somehow) denies them access to the current OpenAI/Anthropic models, they'll be able to improve based on what they already have.
I know this isn't exactly a scientific test, but I had a local Qwen 3.6 27B model implement a fairly sizable feature today. There were a couple of bugs, mostly around me not giving sufficient specifications, but they were ironed out quickly when I pointed it out. I was able to ask the model to create instructions so next time it doesn't fall into the same pitfalls, and it did a great job. 27B local model! (And it was super fast too).
I ran Fable 5 as a code review and it didn't really have any significant corrections.
I guess my point here is that, for most work the frontier models are probably overkill anyway, and improving on overkill in a way that raises prices significantly is probably not a winning strategy.
The only place I can think of where the super high powered models are "required" is if you want to do a ridiculous token burn like GasTown where you just have it run un-monitored on very long tasks. To me though, that's an experiment, not a real workflow. And the way these labs are like "oh we made this (broken) thing in a week using just agents!" always also follows with "and it cost $100,000+ in tokens!". Like, ok, I get it if you're doing research but that's the salary of an entire person.. that can actually learn and improve.
Actually, that has already happened in many domains, it's just that most western people (USA especially) won't admit it.
at the same time, I don't buy the idea that distillation is unimportant in assessing what Chinese labs are capable of. If it wasn't, why did Kimi's release timing coincide so well with Fable's launch?
and if Anthropic hadn't released Fable, would we have Kimi today? If the answer is no, then I think that's still a very important point to consider.
For some reason folks seem to think that China can take action and then other countries can't also take action or respond to that action and it comes up again and again. China has hypersonic missiles! Pack it up boys time to go home. Nothing we can do. Dang shucks. China distilled American AI models, welp time to just close it all down and let's just write off those trillions of dollars and all the literal geniuses financing and building these things. Oh well China can just copy American models while we spend all the money! Ok we just stop developing models and we'll just copy their models. China will flood the market with their cheap products! Nope can't do anything like, oh, idk, not buy any of those products or just raise the prices on them in local markets. It's never-ending. I don't understand the lack of capacity to reason about other actors that takes commonly takes place. And that's just China, never mind other general issues.
That works both ways, competition and performance spur new developments. You don't think the American labs are looking at Chinese research on how to reduce compute per token?
Let's not forget how much people talked about "prompt engineering" before Deepseek mainstreamed the idea of thinking mode which is now universal
Note that Chinese companies are free to rent from GB300 clouds internationally. There are large datacenter hubs in Singapore and Malaysia serving chinese and other customers.
Though there is also reported [1] significant smuggling of Nvidia chips into China as well.
The word I would use is inevitable. It reminds me of the (PC) clones wars…
Also, companies that use distillation may be competitive but seem unlikely to surpass the companies that are training these models from scratch.
US dominance is also important for approaches to safety, especially political approaches. If the frontier models are all US-based, safety might be tackled via internal US policy. If other countries can independently train competitive models, international cooperation is required.
Edit: It is also important for the business model. Companies won't be able to justify tremendous training costs if competitors can replicate their product much more cheaply via distillation.
I don’t think China’s necessarily any better, but I’d rather have the most powerful models be open rather than under the exclusive control of the US executive.
No, no we do not.
We already know it's false because you would have hundreds of competitors if it was that easy.
The reason why these Chinese labs are releasing good models is simpler, they have access to a tremendous pool of talented people.
Because what they want them to think is "the AI factory has unique proprietary technology that cannot be replicated"
What they don't want them to think is "it's relatively easy once you know the basics to bootstrap to near SOTA and so the commercial case for selling inference has an extremely short profitability horizon with little if any brand loyalty or lock in".
It's showing that 'distillation' is a viable way to reclaim all of what they stole and hoard, and with enough luck their debts will come due in time for them to feel it.
Correct, but it at least helps answer the question of "how do they make such good models for a fraction of the price???" The answer is someone else spends the untold billions and Chinese labs do a little tweaking.
All LLMs are trained on the corpus of humanity's knowledge, the legacy of everyone who's ever lived and our civilization as a whole.
Anything that prevents or circumvents the accumulation or gatekeeping of this knowledge and puts it in the hands of more people (that are not AI company shareholders) is a good thing. Whether that is done by open sourcing the model weights, the training set, or by making the output better and cheaper, it is all fair game and is, as another poster mentioned, inevitable in the long run.
> Distillation is not illegal by every definition of the word
Note that Anthropic (and USG) alleges [0] not only that Kimi was distilled, but that they actively circumvented measures intended to stop distillation. There are multiple ways that's illegal, including:
- Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done.
- Economic espionage: 18 U.S.C. §1831 criminalizes obtaining a trade secret through theft, fraud, or deception while intending that it will benefit a foreign entity.
- Trade-secret misappropriation: if Anthropic could argue industrial-scale querying reconstructed proprietary aspects of Fable (like by showing it produces similar outputs, as others have done) then it's illegal under 18 U.S.C. §1832.
- California computer-access statute §502 bars knowingly accessing a computer system and, without permission, taking, copying, or using its data.
- Computer Fraud and Abuse Act protects against the case where restrictions against an activity are circumvented (like Kimi is alleged to have done).
> There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them.
A lack of prosecution does not make something legal. There is also the scale/commercialization thing, which isn't an issue with random tiny HF datasets/models. Remember: Kimi also sells K3 inference.
> kimi architecture is vastly different than that of fable
How do you know that? Do you work for Anthropic? Also, this has nothing to do with architecture, we are talking about data.
> US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable.
Cool. The difference is that one of those things is legal (because they chose to open-source) and one of those things is illegal theft of trade secrets (because it was stolen).
> Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.
1) this has nothing to do with other labs, just Moonshot (and Z.ai, MiniMax, DS)
2) slandering or not it happens to be completely true, so, there's that
[0] https://www.anthropic.com/news/detecting-and-preventing-dist...
In this way, it is different from literal theft. Stealing money/objects from a thief and keeping them is not justified.
An analogy might be a baker stole 20% of the flour used to bake their special bread, which was then stolen. Both thefts are obviously wrong and bad.
"Judge approves a $1.5B Anthropic settlement over pirated books used to train the Claude chatbot"
https://abcnews.com/Technology/wireStory/judge-approves-15b-...
In this case: resolve the theft claims against the US frontier labs, and only then let them make claims against third parties. It would be totally unreasonable for (say) OpenAI to extract a settlement from Moonshot and use that to pay its own claims. Ordering matters.
Same thing here. This whole situation is just comical.
Now, if that kid were to print the bootleg translation and sell it to schoolmates, that's worth a slap on the wrist. The kids willing to pay would likely have paid for official copies.
When these LLM labs download our works, feed them into their models, and sell the output to people that used to pay for our work, that's worth a very hard slap. I honestly have less of a problem with the open models.
It seems like you think theft is bad everywhere except when Anthropic does it.
This is true, but Kimi also has a variety of defenses. Kimi can't raise unclean hands if Anthropic systematically violated others' terms of use, but it can raise copyright misuse (which is similar in some respects to unclean hands) as well as lack of standing to enforce restrictions in the contract due to the third party beneficiary principle (i.e., Kimi would argue that Anthropic cannot sue Kimi for derived IP that rightfully belongs to third parties whose terms of use were violated by Anthropic, and the proper party to sue Kimi, if any, would be those third parties). That latter argument usually fails in small-scale cases (ProCD) but has been successful in larger ones where the alternative would be anticompetitive.
When I said "Does this matter?" I specially meant that distillation in itself, the data you get from distillation is first and foremost not owned by anthropic nor is it copyrightable. If a user willingly gives up their anthropic reasoning data/traces that is 100% legal no matter what the "terms of service" say as it's not enforceable and would fall apart in court.
And what I explicitely pointed out that focusing so much on distillation is an attack on open research and claiming that the majority of advancements are thanks to US labs which is simply not true (at least not anymore this was somewhat true during deepseek R1 era), but that in itself was inspired by open research.
> How do you know that? Do you work for Anthropic? Also, this has nothing to do with architecture, we are talking about data.
Because anthropic would be the first ones to make that information public and the architecture is unique to kimi... They made it, they wrote papers on it, it's their research.
P.S. none of the quoted laws apply here since no trade information is stolen, the one about circumventing distillation protection might hold up in court although unlikely.
Agree, and this is exactly what Anthropic is alleging.
> data you get from distillation is first and foremost not owned by anthropic nor is it copyrightable. If a user willingly gives up their anthropic reasoning data/traces that is 100% legal no matter what the "terms of service" say as it's not enforceable and would fall apart in court.
It's important to note this is NOT what happened. Anthropic was able to trace data directly back to employees at the company: "We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff."
> none of the quoted laws apply here since no trade information is stolen
There is a lot of work showing Kimi models produce similar outputs to Anthropic models, which constitutes trade information. This is not dissimilar to past and ongoing IP suits against Anthropic and OpenAI by showing the models would recreate images of Mickey Mouse/NYT articles etc.
For the record, I'm a researcher myself and I'm well aware how competent the researchers are at the open-source labs/how much they've contributed. But that's not at issue here, my disagreement with you is specific to your arguments about legality; you're conflating what you think should be legal with what actually is legal.
Everything else is simply justifying why it shouldn't, the specifics don't really matter as there is no legal framework to stop china from continuing to distill models and anthropic has proven they cannot use software solutions to stop it either as distillation is still a problem. But I do still believe it wouldn't hold up in court either way as stopping companies from generating training data which was trained on the entire human knowledge corpus is just stealing from thieves and making it 'open' once again so the argument only gets weaker.
Ah yes, I remember when Anthropic crawlers abided by the TOS of the websites they slurped up.
All your other points are downstream from this, which makes them pretty tenuous. Labs don't think that ToS or other explicit wishes of content providers apply to them, but they expect everyone else to abide by theirs.
US and CA law really don't care that Anthropic violated IP law elsewhere.
Plainly who gives a flying fuck. The US can claim whatever rules they want and so can China or any other country. On international level all those rules are artificial constructs unless they can be enforced. China can just say for example that they do not recognize copyrights /patents / whatever so it is "legal" for them.
1) its not illegal (it is)
2) it shouldn't be illegal because Anthropic stole training data (thats not how the law works)
I am a practical man. From what I see laws are mostly for common folks and often do not even serve real justice. The higher one goes and the amount of money / power involved the more the laws bend and on international level the only law that matters is the size of one's club and willingness to use it. And when the country with supposedly biggest one starts crying I find it laughable.
The reason why the United States government is weighing in is because it's in the national interest of the US to have supremacy in "AI".
Legality or lack thereof is one of many data points about whether a thing is noteworthy.
Moonshot performing distillation is rational from their point of view. Reducing costs is in the interest of businesses. It's also rational for frontier labs and the US government to add obstacles to this process.
As consumers this is probably a positive development.
And OpenAI scraped and distilled that answer and gave me nothing
I would prefer some sort of democratiziation of the money made from the democratization of information as well
Circumventing costs.
I mainly focus on the last.
It will be hard for a frontier lab to justify spending the compute and data curation needed to advance AI further if that expenditure can be assimilated into your competitor's products within months/weeks. So reality will present labs with three choices:
A. Cease spending massive amounts of money and compute improving those models.
B. make those improved models more difficult to distill from, either through some regulatory regime, or some technical solution, which seems unlikely to me.
C. making the best models available only to select partners and government.
In all these potential outcomes, China, which lacks compute that U.S. labs enjoy, will likely stop seeing massive improvements in their AI models. Improvements to be sure, but right now they are enjoying gains from distillation AND their own model innovations, and these potential outcomes would largely stop one of those sources.
part of the definition of theft is that the original owner is deprived of it, which does not apply to copyright infringement.
You can only argue with damages from the perspective of potential profits, still not theft though.
I think you're wrong: there is absolutely damage to the authors and publishers from what the AI companies have done.
Or with services, if a barber cuts your hair and then you run away without paying them, do you not consider that theft, even though there's no change in ownership occurring?
I mean honestly if they did that why should I care? I'm happy to see copyright violated in a manner that leads to the creation of new technology. IP law exists strictly for the benefit of society and by all appearances AI is an incredibly powerful tool.
Also while I'm at it libgen is a gift to humanity. Information wants to be free. Spreading and preserving knowledge is generally one of the most wholesome activities anyone can undertake as far as I'm concerned.