undefined

points

[-]

And this is really how it should be. Honestly the only thing I want arxiv to do is become more like open review. Allow comments by peers and some better linking to data and project pages.

It works for physics because physicists are very rigorous. So papers don't change very much. It also works for ML because everyone is moving very fast that it's closer to doing open research. Sloppier, but as long as the readers are other experts then it's generally fine.

I think research should really just be open. It helps everyone. The AI slop and mass publishing is exploiting our laziness; evaluating people on quantity rather than quality. I'm not sure why people are so resistant to making this change. Yes, it's harder, but it has a lot of benefits. And at the end of the day it doesn't matter if a paper is generated if it's actually a quality paper (not in just how it reads, but the actual research). Slop is slop and we shouldn't want slop regardless. But if we evaluate on quality and everything is open it becomes much easier to figure out who is producing slop, collision rings, plagiarist rings, and all that. A little extra work for a lot of benefits. But we seem to be willing to put in a lot of work to avoid doing more work

by abdullahkhalids2 hours ago|

parent|

[-]

I don't agree actually that is how it should or can work for everyone. Senior researchers produce good quality research, and they have a network of high quality peers built over decades. Both those are necessary for them to reach out and ask for feedback, and get genuine and high quality feedback.

Junior researchers don't have these typically. They also benefit more from anonymous feedback, which enables the reviewers to bluntly identify wrong or close to wrong results. So I think open journals should continue to exist. They fill an essential role in the scientific ecosystem.

by lokar2 hours ago|

parent|

prev|

[-]

You could imagine separating the "publishing" part, which really should just be open with minimal anti-spam etc, from the "this was reviewed by a trusted group of people so you should give it more consideration" part. You could do the second without it being attached to the publishing.

by godelski26 minutes ago|

parent|

[-]

I think your phrasing was good. A lot of people conflate a work being published is equivalent to peer reviewed and that "peer reviewed" means "correct".

I think when you think about publishing as what it actually is, researchers communicating to researchers, what I said makes much more sense. I do think formal review does help reduce slop but I think anyone who has published anything is also very aware of how noisy the system is and how good works get rejected or delayed because they aren't "novel" enough.

Honestly, my ideal system is journals with low bars. We forget this prestige bullshit and silliness of novelty (often it's novel to niche experts but not to others) and basically check if it looks like due diligence was done, there's not things obviously wrong, no obvious plagiarism, and then maybe a little back and forth to help communicate. But I think we've gotten too lost in this idea of needing to punish fast and that it has to be important. Important to who? Tons of stuff is only considered important later, we've got a long track record of not being so great at that. But we have a long track record of at least some people working on what we later find out is important.