Impact Factors and the Central Limit Theorem: Why Citation Averages Are Scale Dependent

03/06/2018
by   Manolis Antonoyiannakis, et al.
0

Citation averages, and Impact Factors (IFs) in particular, are sensitive to sample size. We apply the Central Limit Theorem (CLT) to IFs to understand their scale-dependent behavior. For a journal of n randomly selected papers from a population of all papers, we expect from the CLT that its IF fluctuates around the population average μ, and spans a range of values proportional to σ/√(n), where σ^2 is the variance of the population's citation distribution. The 1/√(n) dependence has profound implications for IF rankings: The larger a journal, the narrower the range around μ where its IF lies. IF rankings therefore allocate an unfair advantage to smaller journals in the high IF ranks, and to larger journals in the low IF ranks. We expect a scale-dependent stratification of journals in IF rankings, whereby small journals occupy top, middle, and bottom ranks; mid-sized journals occupy middle ranks; and very large journals have IFs that asymptotically approach μ. We confirm these predictions by analyzing (i) 166,498 IF & journal-size data pairs in the 1997--2016 Journal Citation Reports of Clarivate Analytics, (ii) the top-cited portion of 276,000 physics papers published in 2014--2015, and (iii) the citation distributions of an arbitrarily sampled list of physics journals. We conclude that the CLT is a good predictor of the IF range of actual journals, while sustained deviations from its predictions are a mark of true, non-random, citation impact. IF rankings are thus misleading unless one compares like-sized journals or adjusts for these effects. We propose the Φ index, a rescaled IF adjusted for size, which can be generalized to account also for different citation practices across research fields. Our methodology applies also to citation averages used to compare research fields, university departments or countries in various rankings.

READ FULL TEXT
research
03/06/2018

Impact Factors and the Central Limit Theorem

In rankings by average metrics, smaller samples are more volatile: They ...
research
11/05/2019

Impact Factor volatility to a single paper: A comprehensive analysis of 11639 journals

We study how a single paper affects the Impact Factor (IF) by analyzing ...
research
11/05/2019

Impact Factor volatility to a single paper: A comprehensive analysis

We study how a single paper affects the Impact Factor (IF) by analyzing ...
research
06/06/2019

How a Single Paper Affects the Impact Factor: Implications for Scholarly Publishing

Because the Impact Factor (IF) is an average quantity and most journals ...
research
06/29/2021

When standard network measures fail to rank journals: A theoretical and empirical analysis

Journal rankings are widely used and are often based on citation data in...
research
12/16/2021

Citation inequity and gendered citation practices in contemporary physics

The historical and contemporary under-attribution of women's contributio...
research
11/30/2022

Measuring the Competitive Pressure of Academic Journals and the Competitive Intensity within Subjects

A journal's impact and similarity with rivals is closely related to its ...

Please sign up or login with your details

Forgot password? Click here to reset