I did five, then gave up and just pressed A until I reached the end. I got 3/5 right.
replyExactly the same here. 4/5.
replyYea, I did two and then harrumphed in annoyance that I was expected to do all 10.
replyWe must do something about this immediately! Immediately! Immediately!
reply> Telling me immediately when I got one wrong lets me recognize patterns and improve my guesses.
replyWouldn't this make it a worse measurement?
Well, the alternative is a lot of us spammed A to get to the end, so the data quality is already horrendous.
replyYes. Did one - saw that I wouldn’t get feedback until I have completed all 10 (if at all) and noped out.
replyI believe this is to test the theory that people can detect watermarked text.
replyTeaching you how to identify watermarked text while the experiment is running would ruin the data.
Maybe detecting watermarked text is a skill to be attained. Not allowing proper feedback and training will not allow people to notice the difference on time, thus ruining the data?
replyBest practice is to allow a number (scaled based on complexity of task) of training rounds (with short feedback loops) prior to letting people loose on the regular samples.
Detecting watermarked text is hard. Detecting AI garbage text is easy but then classfiying the garbage further is beyond us.
reply