upvote
Can you really reproduce it though?

I thought it’s the experiments that have to be able to reproduce, not the literature review

reply
Whether you can or can't in reality is moot, unfortunately. The literature search in biomedical fields should indeed be theoretically reproducible. I don't know about other fields, but it would seem odd to me if a search was not reproducible, that would lead to a very arbitrary literature selection.

As for the experiments, yes, in experimental fields. But in all (most?) fields, including non-experimental, the whole process should be well documented so it could be reproduced end-to-end if possible. If it's not reproducible there should be good, well explained reasons why not.

Note that reproduciblity does not necessarily mean the exact same answer will definitely emerge, just that the methods can be followed closely.

reply
Got that, thanks for the advice, I'll ask my supervisor how to address that properly
reply
Yes, you must be able to reproduce the results. If reproduction is not possible, the work lacks scientific validity.
reply
> Any literature review must be reproducible.

That's totally at odds with my understanding, but perhaps this differs between fields.

reply
Quite probably there are differences between fields. In biomedical literature reviews the search terms and databases are detailed, and (in systematic reviews) a PRISMA flowchart [0] provided. The theory being that other researchers could repeat the searches and the in/out decisions and get the same stack of papers to review.

[0] https://www.prisma-statement.org/prisma-2020-flow-diagram

reply
Okay yeah that sounds closer to what I'd call a meta-analysis. In linguistics (which is the field I was in) "literature review" just means "someone looked around and read some papers they thought might be related". There's no expectation that it will be systematic in any replicable way.
reply