ELO is like probability. A 100 ELO difference is something like 2x odds of winning a game and over a lot of tries it tends to be quite accurate. That was also an argument from Magnus Carlsen that to compare human ELO you need a lot of games and he didn't want to take part in classical world championship quoting this reason (among others).