I keep hearing this all the time, and I think this is an absolutely massive overstatement that you cannot go back to 60 hz after experiencing high refresh rates.
My personal experience is that I do notice the difference - for like a minute - and then I adjust and never care anymore.