ELO is a bad fit because it requires competition between submitters; but if the idea is interpreted as “contributor karma score” or similar (not everyone’s familiar with the mathematical nature of ELO), then the way to close the loophole is to only consider voting inputs from the human project owner. This project chose to have people lie to a webform rather than lie to a git interface about using AI, so I don’t expect it will be particularly successful at inhibiting AI use by project-involved humans, but certainly it’ll squelch a lot of noise from unattended/passersby.

fix this problem by make the rating value tied to some paid currency - a repo owner would have to pay for the PR, and that PR contributor will now have more currency than previously. In order to have said currency to pay, the repo owner would need to have contributed to another repo whose owner have currency.

The totality of someone's currency is their reputation.

Of course, now the decision becomes...who is the central currency issuer that creates it?

>what happens if a single AI manages to get through to contributor

Then they'll get removed by the humans? Its about cutting down work, not about eliminating the work entirely

The current approach removes about 99% of their overhead it would seem. If they have to do a few manual interventions here and there, that seems like a huge win overall

Reputation scores, review cartels. This all sounds familiar!

contributors being able to grant contributor to other users seems like a problem