The work from Anthropic isn't a new idea, but they made the process more efficient, as inference verification is one of the biggest bottlenecks in the AI lifecycle. Beyond that, there's a lot more that can be proven, like showing a model was trained on a specific dataset with certain properties.
From an auditing perspective, this used to be handled either by showing the data (which is pretty rare), checking samples (which has been shown to be easily hackable), or similar workarounds. For inference, an auditor might want to interact with a model on demand and have absolute certainty that the model they're talking to is actually the one declared, and not a substitute.
Pleased to hear somebody cares about this. What is it a bottleneck for though? Are people actually doing this right now in non-academic settings? I had the impression this is not in fact usable with larger models.
The bottleneck has two sides: first, there's a technical bottleneck. ZKPs are quite expensive, and finding the right, production-ready protocol is challenging. The other side is a socio-technical bottleneck. There is always quite a large friction with adopting these kind of technologies, also due to the first bottleneck.
As you said, it is difficult to adopt because of how fast the complexity grows with big models. It's great indeed to see that Anthropic is tackling the challenge.
but why do I care about that? can you some example "user stories"?