One of the things I'm/we're thinking through is how to do this in a way that’s participatory rather than extractive. It's not necessary that what scientists would choose to automate with agents is the same as what a company trying to automate scientists would choose.

If we can help people automate drudgery while giving them a real say in what gets built and control over how their work is used, I think there’s a possibility of a much better outcome than the current trajectory. I’m not sure we’re in that world, but I think we should try.

They're already kinda protected, open weight releases rarely come with datasets, especially post-training datasets. NVIDIA is probably the exception.

I also wonder when the NSA starts sharing their wiretaps/etc with labs for 'national security'. Probably already happening, to be honest.

I'm not sure I'll do it justice, but one of the general ideas behind open-model initiatives is that the economic impacts of specialized post-training are a fait accompli. OpenAI and Anthropic have spent $many-millions procuring closed posttraining data.

The relevant question here is whether you want open-weight models — projects like Open Athena — to remain or become competitive in the same range of specialties.