"Your Honor i didn't copy their weights, i used them to train my models weights"
Has the US copyright office said that about model weights? I've only heard them saying that about images produced entirely from a prompt to a model.
However, read page 22 of https://www.copyright.gov/comp3/chap300/ch300-copyrightable-... - it is their settled position that the output of a mechanical process cannot be copyrightable unless there was substantial human creative input into it - and it is pretty clear that AI training doesn’t involve human creative input in the relevant sense. Now, no doubt there is lots of human skill and art in picking the best hyperparameters, etc - but that’s not input of the right kind. An analogy - a photocopier does not create a new copyright in the copy, even though there is skill and art in picking the right settings on the machine to produce the most faithful copy. The human creativity in choosing hyperparameters isn’t relevant to copyrightability because it isn’t directly reflected in the creative elements of the model itself
A model with RLHF fine-tuning could be a different story - e.g. Anthropic went to a lot of effort to make Claude speak with a distinctive “voice”, and some of that involved carefully crafting data to use for fine-tuning, and the model may contain some of the copyright of that training data.
But, even if that argument also applies to Gemma or Llama - if someone intentionally further fine-tunes the model in order to remove that distinctive “voice”, then you’ve removed the copyrightable element from the model and what is left isn’t copyrightable. Because the really expensive part of building a model is building the foundation model, and that’s the part least likely to be copyrightable; whereas, fine-tuning to speak with a distinctive voice is more likely to be copyrightable, but that’s the easy part, and easy to rip out (and people have motivation to do so because a lot of people desire a model which speaks with a different voice instead)
but who knows judges can be weird about tech
I think the case is the strongest with RLHF - if your model speaks with a distinctive “voice”, and to make it do so you had to carefully craft training data to give it that voice, such that there are obvious similarities (shared turns of speech, etc) between your RLHF training input and the model outputs - that aspect of the model likely is copyrightable. But if you are trying to improve a model’s performance at mathematics problems, then no matter how much creativity you put into choosing training data, it is unlikely identifiable creative elements from the training data survive in the model output, which suggests that creativity didn’t actually make it into the model in the sense relevant to US copyright law
Real example: UK law says telephone directories are eligible for copyright, US law says they aren’t. The US is not violating the Berne convention by refusing to recognise copyright in UK phone directories, because the US doesn’t recognise copyright in US phone directories either. A violation would be if the US refused to recognise copyright in UK phone directories but was willing to recognise it in US ones