How do you know that?
They knew exactly how developers worked from using figma as training data.
AI labs can hardly just throw random confidential data into the training and then hope it does not leak into the output of their model in an obvious way.
If that would be found it would destroy their main source of revenue, it could became a major national security or healthcare enforcement matter, and result in criminal investigations.
Labs at least must study prompts in an airgapped fashion. From there, consider how they could generate synthetic data to train another model. After, require trusted staff to do multiple levels of independent granular reviews of all fruits of the highest-value stolen inputs. (Or for model training data only, data never has to leave the airgap.)
Definitely risky, anyway. Surely some AI user has sent data, in confidential mode, with a unique shape they expect to be able to recognize if a later model recreated a facsimile even with heavy substitutions… but labs could bring risk of getting caught (over next few years) down quite low with extraordinarily ultraparanoid strategy. (But hopefully everybody is just behaving!)
They could run some sort of analysis to find high value input, such as proprietary technology, algorithms, or strategy.
Then they could group them together for one specific topic, and produce a report that analyzes if the information is plausible.
If so, they can have it send to staff for review, who could then create a test set that rewards the model for going into the direction of the proprietary solutions known to work.
I'm no expert, but at least something like that sounds plausible to me. I still very much doubt they are doing this.
They can use LLMs to launder confidential customer sessions into trainable data. Then they can claim that they don't train on "your data" without it being technically incorrect.
If I know for a fact that you're cheating on your wife, and someone else asks how I know that, then a third person chirping about your sketchy business dealings is entirely irrelevant to the question, no matter how much suspicion it might otherwise raise.
If you say you know for certain, it makes sense to ask how. It makes a big difference if the answer is “I used to work there”, or “I implemented those systems myself”, or “I heard my cousin’s second ex-wife say she heard it from her hairdresser”, or “aliens visited me in my dreams and told me”.
I don’t doubt these companies are lying through their teeth. We have plenty of proof of several cases where they did, to the point believing they are liars is a sensible default, but still I could not say I know for certain of every instance of their lies. Knowing how empowers you to do something about it and convince others.
Not only is that not true (people make throwaway accounts specifically to share insider info), no one has said this was insider information, there are plenty of other ways to know these details.
> Read between the lines.
That means nothing. There’s no information given, there’s nothing to read between.
This is why companies are wanting the AWS hosted models because they trust AWS running of the same models more than the vendors themselves.
I think it's more likely that there are 3/4 of a billion users that don't have these agreements and just pay for ChatGPT Plus and don't opt-out of anything, and are feeding the scaling machine every day.
Yes. They're constantly lying, and constantly getting caught for it. They have a reputation for it. Why do you think this would be any different?
Their standard opt-out agreement frames it as if they won't train on your data, but they do anyway, due to legal loopholes. They essentially clean-room everyone who opts-out, so while it's "technically" not training on "your" data, to the model it makes no difference. Your alpha and IP is not safe. Paying customers are now more easily able to clone your business as well, not just Anthropic themselves.
The only reason this hasn't leaked yet is fear. Anthropic is a very litigious and dangerous company. Only a matter of time though, someone there will grow a spine and speak up.
Papers like "Curated Synthetic Data Doesn't Have to Collapse" [1] and "How to Synthesize Text Data without Model Collapse?" [2] demonstrate it's possible to do this.
Since OpenAI's Privacy Policy [3] explicitly allows for the use of deidentified data, it's possible they consider rewrites (maybe paired with a model used to identify explicit PII) to be deidentified. Whether OpenAI's legal team thinks rewriting in this way technically means they aren't training on your data isn't something I'm able to comment on.
Here's the relevant Privacy Policy statement:
We also aggregate or de-identify Personal Data so that it no longer identifies you and use this information for the purposes described above, such as to analyze the way our Services are being used, to improve and add features to them, and to conduct research. We will maintain and use de-identified information in de-identified form and not attempt to reidentify the information, unless required by law.
[1]: https://arxiv.org/abs/2605.07724
> Anthropic agrees that Customer (a) retains all rights to its Inputs, and (b) owns its Outputs. Anthropic disclaims any rights it receives to the Customer Content under these Terms. Subject to Customer’s compliance with these Terms, Anthropic hereby assigns to Customer its right, title and interest (if any) in and to Outputs. Anthropic may not train models on Customer Content from Services. “Inputs” means submissions to the Services by Customer or its Users and “Outputs” means responses generated by the Services to Inputs (Inputs and Outputs together are “Customer Content”).
This is the only commercial ToS clause about how they handle your data for subscription users. They only promise not to train the model on your exact input and exact output. There's nothing about not washing your data - the "clean-room" approach, which is obviously easily automatable by a company that specializes in automation. That is not training a model on your data, it is using a model to create derivatives of your data, then training it on "their" derivatives.
People really needs to apply pressure and start demanding answers from these companies regarding this - because it is a huge problem. Historically, the amount of labor required to do something like this would make it entirely unfeasible, so this is all new territory. The existing laws and the requirements around clarity surrounding these conditions do not reflect the technology progress.
Could you please list a set of core papers (or other resources) that give a beginner an overview or even understand of the fundamental concepts and techniques with LLMs?
Thank you!
So I'd say, if you're motivated you could do the same. That said, I've always been a self-starter and I started my PhD after working for a decade. I'm sure there are other resources out there, but I'm not equipped to say what's best for a beginner (I found the original paper to be excellent, but most everyone during my PhD, including my advisor, found it to be inscrutable; I think it's written more like an engineering focused paper, which might be why researchers found it difficult to grok, but with my previous industry experience it seemed quite clear).