I agree that they are historically important, but if you want to query old thinking in 2070, you'd probably be better off having a modern model analyze archive.org. If that ever goes down, we're sunk.

> If you wanted to ask a 2002 person what they thought about future historical events you can just ask them directly.

The weights arent the truth tho, maybe a timecapsule-vhs but i wouldnt trust llm weights more than more hardcore deterministic media that might get preserved to infer facts from an era.

The companies doing the training are becoming the "winners" that are "rewriting history" as they train their models.