upvote
If you are refering in terms of the code of practice part of the EU AI Act

> I've read that watermarking should in theory be impossible to detect except by the entity that watermarked it

This is a carveout exception, for watermarking. In the spirit of those terms it should be machine identifiable.

In my opinion they should have thought better about this, paricularly for text, because in its current forms it is easy to lead next to a new "tamper-proof" requirement, which in practice is DRM. And we do not need more DRM.

For images, music there is metadata already where such information can be stored. And if end users are found using unlabeled AI their accounts could be ban from these platforms. Not something the social platforms might want, but it's a saner approach than trying to reinvent the secret printer dots on all generated media.

reply
Watermarking only applies where the AI has a free choice (https://www.anthropic.com/news/claude-text-watermark).
reply
i think practical question is whether the detector survives ordinary transformations of the text. if for say i paraphrase a watermarked answer with another model, do we expect the original signal to disappear and the second model's signal to replace it?
reply
There are multiple ways to paraphrase and the paraphrasing model is going to use its own random number generator whenever it thinks there's more than one possible choice. (Not really binary; the RNG will have more or less effect.)

So it will definitely be watermarked by the paraphrasing model. But the question is whether the original signal survives at all. There might be a weak signal that's detectable with enough text?

reply
If its paraphrased to any significant degree I'd expect the original to not survive. The second model's watermark would of course be there regardless.
reply
Does it still apply with zero entropy?
reply
No. Anthropic's example is completing the sentence "Isaac Newton's most famous work was called Principia ..." has only one correct answer, so nothing to watermark.
reply
deleted
reply
The watermarking is an inextricable part of the token generation, it just using a known pseudo random sequence for the sampling. It’s not a transform that can be applied later.
reply
This obviously stops working as soon as you don't have the entire context. To reliably detect a subset of the LLM's output you need to do something more sophisticated but also more invasive.
reply