> according to Anthropic and Google, it doesn’t really impact the quality of responses. I find that a bit hard to believe, although those guys are much smarter than I.
replyI would guess "it doesn't impact the quality of responses" was guaranteed to be claimed before they even implemented any of the watermarking.
And would come from marketing, not the people who implemented it.
Yeah, it’s hard to believe, specially when you are coding and there’s only one best way to do things, unless it plays with variable naming, or comments
replyWell the algorithm only increases the chances of certain words being chosen/not chosen, rather than guaranteeing it. If there’s a clear answer then that nudge won’t do anything.
replyIf the model’s most recent output is “for (let i = 0; ”, the likelihood of the next token being “i” is probably millions of times greater than any other possible token. Thus even if “i” is on the red list and has its likelihood decreased, it’s not going to suddenly choose another word.
Put another way, on low-entropy tasks like coding, this style of fingerprinting is less effective and needs bigger sample sizes to be recognizable.
That said, even small changes can dramatically affect output quality, which is why I’m still a skeptic.
One way to watermark (assuming temperature is otherwise positive) would be to output the most likely (or optimal) token every so often.
replyBut wouldn't you have to know the exact context before this token in order to verify the watermark? I.e. a paragraph wouldn't be sufficient; you would need the system prompt, previous prompts, and even hidden thinking?
reply