Field- and time-normalization of zero-inflated data: An empirical analysis using citation and Twitter data

12/22/2017
by   Robin Haunschild, et al.
0

Thelwall (2017a, 2017b) proposed a new family of field- and time-normalized indicators, which is intended for sparse data. These indicators are based on units of analysis (e.g., institutions) rather than on the paper level. They compare the proportion of mentioned papers (e.g., on Twitter) of a unit with the proportion of mentioned papers in the corresponding fields and publication years (the expected values). We propose a new indicator (Mantel-Haenszel quotient, MHq) for the indicator family. The MHq goes back to the MH analysis. This analysis is an established method, which can be used to pool the data from several 2x2 cross tables based on different subgroups. We investigate (using citations and assessments by peers, i.e., F1000Prime recommendations) whether the indicator family (including the MHq) can distinguish between quality levels defined by the assessments of peers. Thus, we test the convergent validity. We find that the MHq is able to distinguish between quality levels (in most cases) while other indicators of the family are not. Since our study approves the MHq as a convergent valid indicator, we apply the MHq to four different Twitter groups as defined by the company Altmetric (e.g., science communicators). Our results show that there is a weak relationship between all four Twitter groups and scientific quality, much weaker than between citations and scientific quality. Therefore, our results discourage the use of Twitter counts in research evaluation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/22/2017

Field- and time-normalization of data with many zeros: An empirical analysis using citation and Twitter data

Thelwall (2017a, 2017b) proposed a new family of field- and time-normali...
research
12/06/2017

Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data

Recently, two new indicators (Equalized Mean-based Normalized Proportion...
research
04/07/2017

Normalization of zero-inflated data: An empirical analysis of a new indicator family

Recently, two new indicators (Equalized Mean-based Normalized Proportion...
research
03/17/2023

Altmetrics can capture research evidence: a study across types of studies in COVID-19 literature

There has been a proliferation of descriptive for COVID-19 papers using ...
research
11/20/2019

Do disruption index indicators measure what they propose to measure? The comparison of several indicator variants with assessments by peers

Recently, Wu, Wang, and Evans (2019) and Bu, Waltman, and Huang (2019) p...
research
12/28/2020

On a Japanese Subjective Well-Being Indicator Based on Twitter data

This study presents for the first time the SWB-J index, a subjective wel...
research
03/03/2023

a q-EW-TOPSIS model of grey correlation for supply capacity evaluation

The paper describes a new supply capacity evaluation model based on the ...

Please sign up or login with your details

Forgot password? Click here to reset