Faculty citation measures are highly correlated with peer assessment of computer science doctoral programs

08/17/2017
by   Slobodan Vucetic, et al.
0

We study relationship between peer assessment of quality of U.S. Computer Science (CS) doctoral programs and objective measures of research strength of those programs. In Fall 2016 we collected Google Scholar citation data for 4,352 tenure-track CS faculty from 173 U.S. universities. The citations are measured by the t10 index, which represents the number of citations received by the 10th highest cited paper of a faculty. To measure the research strength of a CS doctoral program we use 2 groups of citation measures. The first group of measures averages t10 of faculty in a program. Pearson correlation of those measures with the peer assessment of U.S. CS doctoral programs published by the U.S. News in 2014 is as high as 0.890. The second group of measures counts the number of well cited faculty in a program. Pearson correlation of those measures with the peer assessment is as high as 0.909. By combining those two groups of measures using linear regression, we create the Scholar score whose Pearson correlation with the peer assessment is 0.933 and which explains 87.2 of the variance in the peer assessment. Our evaluation shows that the highest 62 ranked CS doctoral programs by the U.S. News peer assessment are much higher correlated with the Scholar score than the next 57 ranked programs, indicating the deficiencies of peer assessment of less-known CS programs. Our results also indicate that university reputation might have a sizeable impact on peer assessment of CS doctoral programs. To promote transparency, the raw data and the codes used in this study are made available to research community at http://www.dabi.temple.edu/ vucetic/CSranking/.

READ FULL TEXT

page 6

page 7

page 14

page 15

page 16

page 17

research
01/09/2023

Scholar Ranking 2023: Ranking of Computer Science Departments Based on Faculty Citations

Scholar Ranking 2023 is the second edition of U.S. Computer Science (CS)...
research
10/31/2018

Revisiting the scaling of citations for research assessment

Over the past decade, national research evaluation exercises, traditiona...
research
03/25/2021

Informed peer review for publication assessments: Are improved impact measures worth the hassle?

In this work we ask whether and to what extent applying a predictor of p...
research
06/14/2022

An analysis of retracted papers in Computer Science

Context: The retraction of research papers, for whatever reason, is a gr...
research
06/10/2021

Academics evaluating academics: a methodology to inform the review process on top of open citations

In the past, several works have investigated ways for combining quantita...
research
04/28/2021

Does Publicity in the Science Press Drive Citations?

We study how publicity in the science press, in the form of highlighting...

Please sign up or login with your details

Forgot password? Click here to reset