A better way to interpret this data is to normalize by the vsync interval or the swap chain depth. The slowest config is 2 frames slower than the fastest. At 500Hz this is 4ms extra which is likely imperceptible to everyone but elite pro gamers. At 60Hz this is 34ms extra which is pretty noticeable to even casual gamers.
I certainly want my latency as low as I can get it. But I'm pretty skeptical that anyone is truly feeling the difference of a couple ms.
We notice latency. Neil Peart could almost get sample-precise timing, he was so godlike.
But I wouldn't necessarily say that people can notice it everywhere in every state of mind. The medium, context etc all matter a lot.