The open weight model providers don't provide the training data or the build tooling. You cannot reproduce the model yourself, or even really know what the model contains. They don't even provide high-level catalogues/descriptions of the training data. An improvement would be something like "trained on the entire WWW up to Aug 1 2026". Or "trained on a Wikipedia archive + Anna's Archive and everything we were able to scrape from Github". They don't provide any of this stuff.

I don't mind open-weight models, but they are not open source. It's like bringing home a dog from the rescue and just hoping that it doesn't have a history of biting kids in the face. You just can't know, because you don't know the full history. You can try to add new training (fine tune) to tell it not to bite kids, but that's it.