A Disciplinary View of Changes in Publications' Reference Lists After Peer Review
share
This paper provides insight into the changes manuscripts undergo during peer review, the potential reasons for these changes, and the differences between scientific fields. A growing body of literature is assessing the effect of peer review on manuscripts, however much of this research currently focuses on the social and medical sciences. We matched more than 6,000 preprint-publication pairs across multiple fields and quantified the changes in their reference lists. We also quantified the change in references per full-text section for 565 pairs from PLOS journals. In addition, we conducted manual checks of a randomly chosen sample of 98 pairs to validate our results, and undertook a qualitative analysis based on the context of the reference to investigate the potential reasons for reference changes. We found 10 disciplines, mostly in the natural sciences with high levels of removed references. Methods sections undergo the most relative change in the natural sciences, while in the medical and health sciences, the results and discussion sections underwent the most changes. Our qualitative analysis identified issues with our results due to incomplete preprint reference lists. In addition, we deduced 10 themes for changing references during peer review. This analysis suggested that manuscripts in the natural and medical sciences undergo more extensive reframing of the literature used to situate and interpret the results of studies than the social and agricultural sciences, which are further embedded in the existing literature through peer review. Peer review in engineering tends to focus on methodological details. Our results are useful to the body of literature examining the effectiveness of peer review in fulfilling its intended purposes.
READ FULL TEXT