DeepAI AI Chat
Log In Sign Up

Peer review and citation data in predicting university rankings, a large-scale analysis

05/22/2018
by   David Pride, et al.
The Open University
0

Most Performance-based Research Funding Systems (PRFS) draw on peer review and bibliometric indicators, two different methodologies which are sometimes combined. A common argument against the use of indicators in such research evaluation exercises is their low correlation at the article level with peer review judgments. In this study, we analyse 191,000 papers from 154 higher education institutes which were peer reviewed in a national research evaluation exercise. We combine these data with 6.95 million citations to the original papers. We show that when citation-based indicators are applied at the institutional or departmental level, rather than at the level of individual papers, surprisingly large correlations with peer review judgments can be observed, up to r <= 0.802, n = 37, p < 0.001 for some disciplines. In our evaluation of ranking prediction performance based on citation data, we show we can reduce the mean rank prediction error by 25 This suggests that citation-based indicators are sufficiently aligned with peer review results at the institutional level to be used to lessen the overall burden of peer review on national evaluation exercises leading to considerable cost savings.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/30/2018

Career advancement and scientific performance in universities

Many governments have placed priority on excellence in higher education ...
07/22/2022

Citation models and research evaluation

Citations in science are being studied from several perspectives. On the...
08/10/2018

Systematic analysis of agreement between metrics and peer review in the UK REF

When performing a national research assessment, some countries rely on c...
03/31/2022

Cite-seeing and Reviewing: A Study on Citation Bias in Peer Review

Citations play an important role in researchers' careers as a key factor...
02/05/2018

Can your paper evade the editors axe? Towards an AI assisted peer review system

This work is an exploratory study of how we could progress a step toward...