upvote
There's an important difference between "knowing" and "believing".
reply
This. People repeat stuff they suspect might be happening like it's facts. Would I be surprised? Only a little. Does that mean it's definitely happening? of course not.
reply
Blame copyright law. AI training is very comparable to compiling an inverted index, which is considered transformational even though you can recover the input documents from an inverted index.
reply
To be maximally charitable, while creating large language models has been ruled to be transformative and thus fair use under current copyright law, Anthropic did separately violate copyright law when procuring copyrighted text from LibGen and Pirate Library Mirror. That being said, I agree with you, creating LLMs from copyrighted material is very clearly not a violation of copyright under the current legal regime, so long as you procure the text legally, be that through web scraping or by purchasing books directly. And it’s annoying that people try to muddy the water on this.
reply
I am an author of a couple of books that are part of the Anthropic settlement.
reply
"be that through web scraping or by purchasing books directly" - or by creating synthetic iterations of the inputs your LLM API service receives.
reply
People downvote you like you're being paranoid, but we're literally discussing this in a thread that shows how little respect those companies have for any sort of trade secrets.
reply
How does the allegation make any sense?

AI labs can hardly just throw random confidential data into the training and then hope it does not leak into the output of their model in an obvious way.

If that would be found it would destroy their main source of revenue, it could became a major national security or healthcare enforcement matter, and result in criminal investigations.

reply
Some of the smartest people on the planet all in the same room, data at their fingertips… they randomly add it to the training set?

Labs at least must study prompts in an airgapped fashion. From there, consider how they could generate synthetic data to train another model. After, require trusted staff to do multiple levels of independent granular reviews of all fruits of the highest-value stolen inputs. (Or for model training data only, data never has to leave the airgap.)

Definitely risky, anyway. Surely some AI user has sent data, in confidential mode, with a unique shape they expect to be able to recognize if a later model recreated a facsimile even with heavy substitutions… but labs could bring risk of getting caught (over next few years) down quite low with extraordinarily ultraparanoid strategy. (But hopefully everybody is just behaving!)

reply
That is an interesting thought.

They could run some sort of analysis to find high value input, such as proprietary technology, algorithms, or strategy.

Then they could group them together for one specific topic, and produce a report that analyzes if the information is plausible.

If so, they can have it send to staff for review, who could then create a test set that rewards the model for going into the direction of the proprietary solutions known to work.

I'm no expert, but at least something like that sounds plausible to me. I still very much doubt they are doing this.

reply
It's actually simpler: your code is not the valuable part, it's the telemetry/metadata/conversation surrounding your session that's valuable. Every time you press escape, every time tell Claude/Codex that it's being an idiot, your back-and-forth conversations, etc. "when/why did we fail and how can we improve?" is what they want to know.

They can use LLMs to launder confidential customer sessions into trainable data. Then they can claim that they don't train on "your data" without it being technically incorrect.

reply
Exactly. They can also feed you a stupider model to goad you into handing over more of this training guidance as well. The incentives are aligned for evil behavior. Open source really needs to win this race, or we need much tighter scrutiny on these AI companies.
reply
That's not at all how they do it. They wash the data. The end result is they can steal your IP insights without it being explicitly tied to your business. All of the decisions you made to build your product? Those become the standard suggestions in the next model for people building the same sorts of products. All of that error correcting you did, which in a normal business would be considered hard-earned IP (like in the case of Apple's lawsuit - the "what not to do" is just as valuable as the "what to do"), is now free correction for the next model to produce the perfect result in a one-shot. Now the AI company can commoditize your labor and your industry, or compete against you if they so wish.
reply
I downvoted because the reply has nothing to do with the argument.

If I know for a fact that you're cheating on your wife, and someone else asks how I know that, then a third person chirping about your sketchy business dealings is entirely irrelevant to the question, no matter how much suspicion it might otherwise raise.

reply
[dead]
reply