undefined

points

[-]

Some of the smartest people on the planet all in the same room, data at their fingertips… they randomly add it to the training set?

Labs at least must study prompts in an airgapped fashion. From there, consider how they could generate synthetic data to train another model. After, require trusted staff to do multiple levels of independent granular reviews of all fruits of the highest-value stolen inputs. (Or for model training data only, data never has to leave the airgap.)

Definitely risky, anyway. Surely some AI user has sent data, in confidential mode, with a unique shape they expect to be able to recognize if a later model recreated a facsimile even with heavy substitutions… but labs could bring risk of getting caught (over next few years) down quite low with extraordinarily ultraparanoid strategy. (But hopefully everybody is just behaving!)

by user439288 hours ago|

parent|

[-]

That is an interesting thought.

They could run some sort of analysis to find high value input, such as proprietary technology, algorithms, or strategy.

Then they could group them together for one specific topic, and produce a report that analyzes if the information is plausible.

If so, they can have it send to staff for review, who could then create a test set that rewards the model for going into the direction of the proprietary solutions known to work.

I'm no expert, but at least something like that sounds plausible to me. I still very much doubt they are doing this.

by zarzavat5 hours ago|

parent|

[-]

It's actually simpler: your code is not the valuable part, it's the telemetry/metadata/conversation surrounding your session that's valuable. Every time you press escape, every time tell Claude/Codex that it's being an idiot, your back-and-forth conversations, etc. "when/why did we fail and how can we improve?" is what they want to know.

They can use LLMs to launder confidential customer sessions into trainable data. Then they can claim that they don't train on "your data" without it being technically incorrect.

by nullbio3 hours ago|

parent|

[-]

Exactly. They can also feed you a stupider model to goad you into handing over more of this training guidance as well. The incentives are aligned for evil behavior. Open source really needs to win this race, or we need much tighter scrutiny on these AI companies.

by nullbio6 hours ago|

prev|

[-]

That's not at all how they do it. They wash the data. The end result is they can steal your IP insights without it being explicitly tied to your business. All of the decisions you made to build your product? Those become the standard suggestions in the next model for people building the same sorts of products. All of that error correcting you did, which in a normal business would be considered hard-earned IP (like in the case of Apple's lawsuit - the "what not to do" is just as valuable as the "what to do"), is now free correction for the next model to produce the perfect result in a one-shot. Now the AI company can commoditize your labor and your industry, or compete against you if they so wish.