undefined

points

[-]

The watermark lives in the entropy of sampled outputs. Typical entropy of sampled English text is about 1 bit/token, meaning that a 500-token response from a given model might have 2^500 potential outputs of roughly equal probability. The watermark restricts the sampler to some subset of these - say, 2^400 of them, so chance of accidentally generating a watermarked output is astronomically small (2^-100). As long as the restriction doesn't condition on the content of the samples themselves, the watermark is "non-distortionary": the outputs are all still samples from the model's original distribution, and so will satisfy all the same statistical properties, including things like expected performance on any benchmark or eval you can construct.

In cases where the output has low entropy - eg, you've asked a model to repeat some input text verbatim, or to answer a question that has exactly one correct answer - there will be no randomness for the watermark to hide in, so the output will effectively not be watermarked. Code lives somewhere in the middle: it generally has less entropy-per-token than prose, so would need more tokens to reach a given level of detectability.

There are lots of ways to restrict output samples. The simplest conceptually would be to just use a restricted pool of PRNG seeds, but in practice there are more sophisticated constructions to try to build in robustness to minor edits, allow detectability without needing the original weights and prompt, etc. Google's SynthID paper (https://www.nature.com/articles/s41586-024-08025-4) is a good starting point if you want to understand a recent production-ready method (or you can just ask an LLM to explain it to you).

by keito9 hours ago|

prev|

[-]

You can generate text with/without watermarking and use a detector in this tool that simulates various watermarking techniques (Claude uses SynthID-Text) using a small LLM: https://watermark.keito.me/ (disclaimer: I made it) It doesn't obviously bias the output as much as you might fear, especially in low-entropy text.

by beamy11 hours ago|

prev|

[-]

This is a good explainer: https://magazine.sebastianraschka.com/p/claude-watermarking

by unglaublich11 hours ago|

prev|

[-]

It does change the output, they never said it did not. They said it would not _noticeably_ affect performance.

by Smaug12310 hours ago|

parent|

[-]

It doesn't necessarily change the output distribution; it depends exactly how it's implemented, and Anthropic haven't told us that. Google's original SynthID paper describes how you can do this.

Toy proof-of-concept: Anthropic owns a secret key which is a coin-flip Bernoulli random variable K with p=1/2. You are paying Anthropic to give you X, a Bernoulli random variable with p=1/2. Anthropic changes from their old strategy, "draw from K, then throw it away and flip a coin, each time you ask for a sample", to their new strategy, "draw from K and send it to you". You cannot observe the difference, but Anthropic knows K and so they know when you are repeating its outputs. (Obviously this is a toy example; in reality the distribution is vastly more complicated than Bernoulli, and Anthropic isn't just storing some model outputs to use as K but instead is computing a correlation with a known pseudorandomness source.)

by graboy8 hours ago|

parent|

prev|

[-]

You have a misunderstanding. Watermarking does not bias the responses in any way. How is this possible?

Before: "He leaped at the chance" - 33%. "Jumped at the opportunity" - 66%.

After: "He leaped at the chance" - 33%. "Jumped at the opportunity" - 66%.

But if you refresh your response from Anthropic 100 times:

Before: "Jumped at the opportunity" He leaped at the chance" "Jumped at the opportunity"

After: "He leaped at the chance" "He leaped at the chance" "He leaped at the chance"

The second one is detectable as being watermarked.

davmre has a good explanation that's more in-depth.

by sroussey11 hours ago|

parent|

prev|

[-]

Which is a lie. Or vacuous statement as Claude might say these days.

by anthonyrstevens7 hours ago|

parent|

[-]

How is it a lie?

by sroussey6 hours ago|

parent|

[-]

The performance of the output is poor and I can detect the watermark but only in a domain i am in the middle of, if it writes a summary of an email i can not tell, but in claude code in my codebase i definitely can tell. I am complaining about it!