“You need to get the author to do an oral defense to verify that they wrote the paper.” While I agree with this point, it can’t scale. As with the indirect response, vishing and likely video mimicry are only going to be easier with the next llm release. This leaves requiring each author who submitted a paper to fly to a place just to defend a paper in person (assuming all volunteer reviewers are in the same place).
Are we prepared to sacrifice everything on the altar of scalability?
At any rate, why can't this scale? Go to the nearest accredited university, sit at a terminal they give you and do you interview.
Universities already have the space to do this, for potentially hundreds of defenders per day.
“Universities already have the space to do this, for potentially hundreds of defenders per day.” Universities have space for their own students while they’re still funded for the time, but conferences aren’t tied to universities. Even so if they were, from the author “The fraction of desk rejected papers at TMLR used to be about 6% in 2023 but is now at about 53%.” universities would have to raise funds for this massive increase of slop review. Us conferences uses unpaid volunteers (most part) from those areas for reviews.
If an independent conference were to hold in-person interviews (and maybe fly people out) for 1000s of submissions, where would they hold it and how would they pay for it?
I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on. I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do.
No. Did you "clean this up" with an LLM before posting it?
I hate to go there, but how is that argument different from a thief who argues "Everyone steals. The only difference is some of them claim they do, and some claim they don't"?
> I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on.
Right. Maybe. The problem is that when material has all the tells of LLM generation, do you expect the reader to figure out if the thoughts were the authors or introduced by the LLM?
The value in a human directly communicating with you is that you understand what they are saying; if they misunderstand, you spot their misunderstanding. If they are talking at cross-purposes, you know you are arguing past each other. If they agree with you, you know they agree with you.
That signal is missing when the communication is LLM generated; you message said "Not X, Not Y, just Z", but I can't tell if you even know what X, Y and Z are. f I ask you to give me a definition for them so I can ensure we aren't talking about different things, the LLM response will be "X is $SOMETHING", but I still can't tell if you think that X is $SOMETHING or if you still think that X is $SOMETHING_ELSE while your LLM and I agree that X is $SOMETHING.
Communication from a person tells us something about that person, outside of the message being communicated. If that communication is relayed via an non-invested third party, the missing signal prevents any communication.
But also: When we want to ask a question of a bot and then get an answer from that bot, then we'd have already made a deliberate and rational decision to ask a bot ourselves.
We not need nor want anyone's help on that front; this kind of low-effort help results in an insulting waste of time.
Abject silence would be an improvement in communication quality over having an unwanted third-party conversation with a bot.