Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating other AIs to contributor, and we're off again. There doesn't have to be a purpose to this. Trolls will troll, and trolls armed with AI bots can devote endless energy to doing so. The more you work to keep them out, the more fun it becomes for them.
I wish I had an answer for that problem. But I don't.
You could probably use some kind of pairwise ranking algorithm (like anything based on the Bradley-Terry model) to rate human vs. AI contributions, but that would take a lot of manual effort. Google is using it to (supposedly) improve their searching algorithms. They give testers two different versions and ask them what's better.
The totality of someone's currency is their reputation.
Of course, now the decision becomes...who is the central currency issuer that creates it?
Then they'll get removed by the humans? Its about cutting down work, not about eliminating the work entirely
The current approach removes about 99% of their overhead it would seem. If they have to do a few manual interventions here and there, that seems like a huge win overall
Frontier users: 527,865 Light indexed: 527,865 Ready to queue: 9,083 Fast scores ready: 0 Activity events 24h: 30,266 Fast scores completed 24h: 19,123 Deep jobs completed 24h: 3,043 Fast-score ETA: n/a Deep-hydrate ETA: 69h Stale running jobs: 0 GitHub backpressure jobs: 19,113 High automation signals: 4,608 Medium automation signals: 1,327 Completed jobs: 74,714
Biggest challenge is Github's rate limits. At this pace it will take two more months to have 98% coverage. But after that the maintenance should be quite straight forward.
The Elo rating system doesn't make sense in this context; it's designed around collecting zero sum game results for a given community of players and building a model around it.
There are a lot of political tricks that get used.
What is scary is that one of those kinds of users are malicious state actors. Like North Korea and Russia...