upvote
They also use synthetic generated data to train on too

If it works and it's verified in nine different ways, throw it in the training pile, and then automate the process.

reply
That is how you end up with model collapse which, I assume, we will see more and more of.
reply
There were fears about this 3-4 years ago but they never manifested. All the labs train heavily on synthetic data today
reply
There is absolutely no way to prove that this claim is true because no model advanced enough to do so at a useful level has any fine grained report of its training data that could be used to verify this.
reply
The only reason at this point that there is no way to prove the claim is true is that we cannot inspect the weights, nor query for copyright violations.

I imagine that someone will eventually figure out how to query black-box LLMs for copyright violations without triggering their guardrails. This is an existential threat for anyone running a black-box LLM with stolen data.

reply
We already know how to and have proved it, before the guardrails got "better" (mostly at hiding it). Early versions of Claude and ChatGPT were fed with a paragraph or two of popular fiction and spat out a half dozen paragraphs more, sometimes verbatim. They're significantly lossy compression algorithms. Not judging just based on that, but it's clear that their creators engaged in far more than the already outrageous amounts of copyright infringement that they've admitted and been so gently slapped on the wrist for.
reply
This is trivially true. Why can they work with a tool or API I’ve written or something internal to a company? They definitely have not trained on my local machine from 2020.
reply
I'm pretty sure companies are not throwing away their training data each time they build a model.
reply