but I'm hopeful that some middle ground will be found in the future
Journals themselves should make policies about the extent to which they allow the use of LLMs. In some areas it might be considered more benign than in others.
"low quality, superficial reviews" have always been around. Reviewing is most often an unpaid, thankless job and many times reviewers barely put in the effort.
I trust an LLM to review that the language used in the paper is grammatically correct, but not to evaluate new information for accuracy.
For a more practical approach you need to use proxies: https://zby.github.io/commonplace/articles/what-an-automated...
1) Humans also are trained on a subset of human knowledge. 2)A lot of papers are just about experimenting something, and then applying simple stats. Eg empirical studies, around 1/3rd of published papers. Like, we tried this drug or did this experiment, from a sample size X here are the results. An expert is needed to maybe comment on the conclusion/hypothesis of the underlying suspected mechanism, but LLMs are still very useful on catching bad statistics or p hacking (so so common)
What can't be gotten rid of fast enough is the notion that having written something is meaningful on its own. Making something that looks right was a level above total novice: now it's the floor.
For a start LLMs love LLM generated text, so you are boosting papers people never had any input in.
Secondly, LLMs in my experience are good at small issues, but fail totally at the whole paper being obviously poorly constructed, or clearly fake.