It's about how they took measures against it. Sabotaging the requests is super shady and breaks all other areas of trust in the company their models.
replyAll they had to do was have a simple, transparent output "Sorry, that request is against our terms of service. This session has been terminated"
The hidden safeguard was not against distilling, it was against "frontier" ML research with no indication whatsoever of what "frontier" might mean, but possibly even including research into model safety or alignment. That amounts to deliberately boobytrapping research across an entire legit academic field, which is ridiculously unaligned behavior.
replyThis is the same as saying "well some unaligned countries will use refined nuclear material for energy, too!" lmao.
replyThe vast majority of frontier research is about how to build better models, not about alignment.
And as a matter of fact, there's a lot of meaningful research into how to have different sorts of nuclear material that might be usable for power production but not hidden malicious development. That's the closest analog to "safety" and "alignment" in your scenario.
reply