Collaboration in the Time of COVID: A Scientometric Analysis of Multidisciplinary SARS-CoV-2 Research

08/30/2021
by   Eoghan Cunningham, et al.
0

The novel coronavirus SARS-CoV-2 and the COVID-19 illness it causes have inspired unprecedented levels of multidisciplinary research in an effort to address a generational public health challenge. In this work we conduct a scientometric analysis of COVID-19 research, paying particular attention to the nature of collaboration that this pandemic has fostered among different disciplines. Increased multidisciplinary collaboration has been shown to produce greater scientific impact, albeit with higher co-ordination costs. As such, we consider a collection of over 166,000 COVID-19-related articles to assess the scale and diversity of collaboration in COVID-19 research, which we compare to non-COVID-19 controls before and during the pandemic. We show that COVID-19 research teams are not only significantly smaller than their non-COVID-19 counterparts, but they are also more diverse. Furthermore, we find that COVID-19 research has increased the multidisciplinarity of authors across most scientific fields of study, indicating that COVID-19 has helped to remove some of the barriers that usually exist between disparate disciplines. Finally, we highlight a number of interesting areas of multidisciplinary research during COVID-19, and propose methodologies for visualising the nature of multidisciplinary collaboration, which may have application beyond this pandemic.

READ FULL TEXT VIEW PDF

Authors

page 1

page 8

page 9

page 10

page 11

09/26/2020

Can pandemics transform scientific novelty? Evidence from COVID-19

Scientific novelty is important during the pandemic due to its critical ...
07/30/2020

How Work From Home Affects Collaboration: A Large-Scale Study of Information Workers in a Natural Experiment During COVID-19

The COVID-19 pandemic has had a wide-ranging impact on information worke...
04/13/2022

CWcollab: A Context-Aware Web-Based Collaborative Multimedia System

Remote collaboration tools for conferencing and presentation are gaining...
10/06/2020

Publication Patterns' Changes due to the COVID-19 Pandemic: A longitudinal and short-term scientometric analysis

In recent months the COVID-19 (also known as SARS-CoV-2 and Coronavirus)...
04/25/2022

Discovering changes in birthing narratives during COVID-19

We investigate whether, and if so how, birthing narratives written by ne...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Introduction

The scientific response to the SARS-CoV-2 pandemic has been unprecedented with researchers from several surprising fields — e.g. artificial intelligence

[nguyen_2021], economics [nicola_2020], and particle physics [lustig_2020] — contributing to solving the many and varied clinical and societal challenges arising from the pandemic. As a result, by January 2021, The Allen Institute for AI[allenai] and the World Health Organisation[who] had identified over 166,000 research papers relating to SARS-CoV-2 and the COVID-19 illness it causes, highlighting an unprecedented period of scientific productivity. In this study we analyse this body of work to better understand the scale and nature of the collaboration and fields of study that have defined this research.

The benefits of collaboration during scientific research are well documented and widely accepted and in recent years there has been steady growth in research team size across all scientific disciplines [leahey_2016, youngblood_2018], which has been shown to correlate positively with research impact [lariviere_2014, porter_2008]. Moreover, multidisciplinary science, which brings together researchers from many disparate subject areas has been shown to be among the most successful scientific endeavours [okamura_2019, lariviere_2015]. Indeed, multidisciplinary research has been highlighted as a key enabler when it comes to addressing some of the most complex challenges facing the world today [leahey_2016]. Not surprisingly then, there have been numerous attempts to encourage and promote collaboration and cooperation in the fight against COVID-19: the World Health Organisation maintains a COVID-19 global research database; scientific journals have published explicit calls for teamwork and cooperation [chakraborty_2020, nature_call_for_papers]; in many cases COVID-19 related research has been made freely available to the public and the scientific community; comprehensive datasets have been created and shared; and reports from the International Chamber of Commerce (ICC) and the Organisation for Economic Co-operation and Development (OECD) have argued for international and multidisciplinary collaboration in the response to the pandemic.

Although early studies have found that the pandemic has generated a significant degree of novel collaboration [hook_2020, liu_2021] other research has suggested that COVID-19 research have been less internationally collaborative than expected, compared with recent research from the years immediately prior to the pandemic [fry_2020, hook_2020]. There is also some evidence that COVID-19 teams have been smaller than their pre-2020 counterparts [cai_2021, fry_2020]. Thus, despite calls for greater collaboration, the evidence points to less collaboration in COVID-19 related research, perhaps because of the startup and coordination costs associated with multidisciplinary research [cai_2021, fry_2020, hook_2020] combined with the urgency of the pandemic response that has been needed.

In this study we evaluate the scale and nature of collaboration in COVID-19 research during 2020, using scientometric analysis techniques to analyse COVID and non-COVID publications before (non-COVID) and during (COVID and non-COVID) the pandemic. We determine the nature of collaboration in these datasets using three different collaboration measures: (i) the Collaboration Index (CI) [youngblood_2018]

, to estimate the degree of collaboration in a body of research; (ii)

author multidisciplinarity to estimate the rate at which authors publish in different disciplines; and (iii) team multidisciplinarity to estimate subject diversity across research teams. We find a lower CI for COVID-related research teams, despite an increasing CI trend for non-COVID work, before and during the pandemic, but COVID-related research is associated with higher author multidisciplinarity and more diverse research teams. This research can help us to better understand the nature of the research that has been conducted under pandemic conditions, which may be useful when it comes to coordinating similar large-scale initiatives in the future. Moreover, we develop a number of techniques for exploring the nature of collaborative research, which we believe will be of general interest to academics, research institutions, and funding agencies.

Methods

In this section we describe our methods for evaluating scientific collaboration in COVID-19 research. We describe the data that we use throughout our analysis, and we outline three approaches used to evaluate collaboration activity.

Datasets

The COVID-19 Open Research Dataset (CORD-19) [wang_2020] comprises more than 400,000 scholarly articles, including over 150,000 with full text, all related to COVID-19, SARS-CoV-2, and similar coronaviruses. CORD-19 papers are sourced from PubMed, PubMed Central, bioRxiv, medRxiv, arXiv, and the World Health Organisation’s COVID-19 database. We generate a set COVID-19-related research by excluding articles dated prior to 2020 and the resulting dataset contains CORD-19 metadata for 166,356 research papers containing the terms “COVID”, “COVID-19”, “Coronavirus”, “Corona virus”, “2019-nCoV”, “SARS-CoV”, ”MERS-CoV”, “Severe Acute Respiratory Syndrome” or “Middle East Respiratory Syndrome”. We supplement this metadata with bibliographic information from the Microsoft Academic Graph (MAG) [sinha_2015].

Notably, we use the MAG fields of study (FoS) to categorise research papers. The MAG uses hierarchical topic modelling to identify and assign research topics to individual papers, each of which represents a specific field of study. To date, this approach has identified a hierarchy of over 700,000 topics within the Microsoft Academic Knowledge corpus. In our dataset of 166,356 COVID-19 research articles, the average paper is associated with 9 FoS from different levels in this hierarchy and in total, 65,427 unique fields are represented. To produce a more useful categorisation of articles, we first reduce the number of topics by replacing each field with its parent and then consider topics at two levels in the FoS hierarchy: (i) the 19 FoS at level 0, which we refer to as ’disciplines’, and (ii) the 292 FoS at level 1, which we refer to as ‘sub-disciplines’. In this way, each article is associated with a set of disciplines (e.g. ’Medicine’, ’Physics’, ’Engineering’) and sub-disciplines (e.g. ’Virology’, ’Particle Physics’, ’Electronic Engineering’), which are identified by traversing the FoS hierarchy from the fields originally assigned to the paper.

Authors Articles Fields
pre-2020 6,379,612 4,017,655 283,599
2020-non-COVID 3,200,107 1,205,434 196,409
2020-COVID-related 627,205 166,356 65,427
(a) A summary of the numbers of authors, articles, and fields collected in the three datasets used in this study.
Year Articles
2016 954,174
2017 1,006,394
2018 987,666
2019 1,069,421
2020 1,371,190
(b) Articles per year across all datasets.
Table 1: Dataset summary: Note, all non-COVID research articles contain at least one author who published COVID-related research. Also, of the authors who published COVID-related research, we are able to collect non-COVID research (both pre-2020 and 2020-non-COVID) for 299,046 individuals.

We further extend this dataset with any additional research published by the authors in the COVID-related dataset. Thus, for each author, we include MAG metadata from any available articles dated after 2015. The final dataset consists of metadata for 5,389,445 research papers, which we divide into three distinct groups as follows; see Table 1 with further detail provided in the supplementary materials that accompany this article (Supplementary Table 1-3).

  1. 2020-COVID-related research: the 166,356 COVID-related articles published during the pandemic (2020);

  2. pre-2020 research: 4,017,655 non-COVID related articles published before the pandemic, that is during 2016–2019, inclusive;

  3. 2020-non-COVID research: 1,205,434 non-COVID related articles published during the pandemic period and which are not in the CORD dataset.

Collaboration Index

The Annual Collaboration Index (CI) is defined, for a body of work, as the ratio of the number of authors of co-authored articles to the total number of co-authored articles [youngblood_2018]. Since larger (more collaborative) teams have been shown to be more successful than smaller teams [lariviere_2014, leahey_2016, klug_2014], we can use CI to compare COVID-related research to non-COVID baselines. However, CI is sensitive to the total number of articles in the corpus. Therefore, to facilitate comparison across our COVID and non-COVID baselines we generate a CI distribution for each dataset by re-sampling 50,000 papers 1,000 times, without replacement, from each year, and we calculate the sample distribution for these CI values for each year in our dataset.

Author Multidisciplinarity

To evaluate the multidisciplinarity of individual authors, we consider the extent to which they publish across multiple disciplines, based on a network representation of their publications. An un-weighted bipartite network, populated by research fields and authors, links researchers to subjects (that is, based on the subjects of their publications). A projection of this network produces a dense graph of the 292 sub-disciplines at level 1 in the MAG FoS hierarchy, with two sub-disciplines/fields are linked if an author has published work in both. We refer to this projection as a field of study network. In such a network, the edges between fields are weighted according to the number of authors publishing in both fields. Due to the large number of researchers, and the relatively small number of sub-disciplines, the resulting graph is almost fully connected. Thus, the edge weights are an important way to distinguish between edges. Using the MAG FoS hierarchy, we divide the network nodes into 19 overlapping “communities”, based on their assignment to level 0 fields of study. This facilitates the characterization of the edges in the graph: an edge within a community represents an author publishing in two sub-disciplines within the same parent discipline, while an edge between

communities represents an author publishing in two sub-disciplines from different parent disciplines. For example, if an author publishes research in ’Machine Learning’ and ’Databases’, the resulting edge is considered to be

within the community/discipline of ’Computer Science’. Conversely, if an author publishes in ’Machine Learning’ and ’Radiography’, the resulting edge is considered to be between

the ’Medicine’ and ’Computer Science’ communities. An edge between disciplines may represent either a single piece of interdisciplinary research or an author publishing separate pieces of research in two different disciplines. To evaluate the effect of COVID-19 on author multidisciplinarity, we produce a field of study network for each year in our dataset and calculate the proportion of the total edge weights that exist between communities. In the special case of 2020 we also report the odds ratio achieved when we compare 2019 with non-COVID research in 2020 (i.e., after we remove COVID-19 research from the graph).

Research Team Disciplinary Diversity

In addition to author multidisciplinarity, we also consider the multidisciplinarity of the research teams, by calculating their disciplinary diversity

. To do this we compare the research backgrounds of different authors using publication vectors based on the proportions of a researcher’s work published across different fields

[feng_2020]

. Specifically, we construct publication vectors for authors in our dataset using the 19 MAG disciplines. Thus, an author’s publication vector is a 19-dimensional vector, with each value indicating the proportion of the author’s research published in the corresponding domain. For example, an author who has 50 publications classified under ’Computer Science’, 30 publications classified under ’Mathematics’, and 20 publications classified under ’Biology’ would have a publication vector with values {0.5,0.3,0.2} for the entries corresponding to these disciplines respectively, and zeros elsewhere. By using publication vectors to represent an individual’s research profile, we can quantify the disciplinary diversity of a research team using Equation

1 from [feng_2020].

(1)

Note, in Equation 1 refers to the size of the research team and

is the cosine similarity of the publication vectors for authors

and . The team research similarity score for an article is a normalized sum of the pairwise cosine similarities for all authors of the article.

To evaluate research team disciplinary diversity, we compute the teams’ disciplinary similarity based on publication vectors from pre-2020 research, and we report as the teams’ diversity. The year of the paper is excluded from the publication vector to avoid reducing team diversity with the common publication. As such, team disciplinary diversities for COVID-related research (and non-COVID research from 2020) are calculated from publication vectors which exclude work from 2020. We compare these scores with disciplinary diversity scores for research in 2019 when, similarly, the publication vectors exclude work from 2019 and 2020. As the potential for disciplinary diversity in research teams is limited by the number of team members, we compare diversity by team size.

Case Studies of Multidisciplinarity in COVID-19 Research

The field of study network structure used to calculate author multidisciplinarity encodes relationships between fields of study, with respect to the authors who publish in them. Since these relationships are altered in COVID-related research, we propose a modified network structure to explore the changes to these relationships visually, and to highlight interesting case studies of multidisciplinary research in the COVID-19 literature. In this modified network structure, COVID-related research articles contribute directed edges to the graph, for all sub-disciplines in which the authors publish in their pre-2020 work, and all sub-disciplines which relate to the article. For example, an edge between the pair of sub-disciplines ’Machine Learning’ and ’Radiology’ represents an author who published in the field of ’Machine Learning’ in their pre-2020 work (2016–2019), publishing COVID-19 research in the field of ’Radiology’. We produce networks of this structure from different subsets of COVID-related research articles, which we will visualise using flow diagrams, where the pre-2020 sub-disciplines are on the left and the COVID-related disciplines are on the right.

Results

Research Team Size and Collaboration Index

Figure 1 reports the mean Collaboration Index for the samples of 50,000 research papers taken from each year in the dataset. Mean values for samples of COVID-19 research articles are also included. The Collaboration Index increases year-on-year, indicating a move towards larger research teams. This trend has been noted across many disciplines of academic research [porter_2008, lariviere_2014, leahey_2016].

COVID-19 research presents with a very different CI (approximately 5.6), however, indicating that COVID-19 research teams are significantly smaller than expected for research conducted by the same authors in 2020. This result is robust with respect to re-sampling size and in the supplementary materials that accompany this article (see Supplementary Figure 1) we report comparable results using sample sizes 10,000 and 100,000.

Figure 1: Collaboration Index distributions for samples of 50,000 research articles from different portions of the dataset. 1,000 samples are taken from each year (2016-2020). Collaboration index increases annually,

, and the CI for COVID-19 articles is significantly less that the CI associated with non-COVID 2020 research; in fact the mean COVID-19 CI is 25 standard deviations below the mean of of non-COVID samples taken from 2020. Thus, research teams publishing COVID-19 research are significantly smaller than expected for research teams in 2020 containing the same authors.

Author Multidisciplinary Publication

We quantify author multidisciplinarity in a year of research by measuring the proportion of the total number of edges in an author-FoS network that are between communities (i.e., disciplines). We find that this proportion is increasing slowly over time when we produce FoS networks for each year in our data (). Figure 2 reports the odds ratio effect size when the proportion of the edges that are between communities in a given year is compared with that of the previous year. These scores are reported for each community. In the case of 2020 we also report the odds ratio achieved when we compare 2019 with 2020-non-COVID research i.e., after we remove COVID-19 research from the graph. Figure 2 shows an increase in multidisciplinary publication in 2020 across almost all disciplines. The increase in author multidisciplinarity is much greater when we include COVID-19 research in the graph. Despite representing less than 20% of the work published in 2020, COVID-19 research contributes greatly to the proportion inter-disciplinary edges in the FoS network.

Figure 2: Odds ratio effect sizes for the proportion of links between disciplines when compared with the previous year. A score of 1 indicates that authors are no more likely to publish in other disciplines than they were in the previous year.

Research Team Disciplinary Diversity

When we compare authors by their publication backgrounds, encoded as publication vectors, we find COVID-19 research teams to be more diverse than equivalently-sized research teams who published before 2020. Figure 3 presents the relative increase in mean research team disciplinary diversity for different team sizes, when research teams from 2020 are compared with teams from 2019. We divide 2020 research into two sets: (i) 2020-COVID-related; (ii) 2020-non-COVID research and report relative increases in team diversity for each set. Independent t tests show COVID-19 research teams to be significantly more diverse than both pre-2020 and 2020-non-COVID research teams of the same size (, see Supplemental Table 4).

Figure 3: Percentage increase in mean research team diversity for research published in 2020, for teams of increasing size, compared to research published by the same authors in 2019. The distribution of research team sizes is also shown.

Discussion

Despite the recent trend towards larger, more collaborative research teams [porter_2008, lariviere_2014, leahey_2016, feng_2020], COVID-19 research appears to have significantly fewer authors than other publications by the same researchers, during 2020. This may be a concerning finding amid evidence that larger teams produce more impactful scientific research [lariviere_2014]: it may have limited the value of the research produced, notwithstanding the incredible achievements that have been made, or it may be a reality of working under the constraints of a global pandemic.

We do see some examples of larger teams and their greater potential for research impact in our analysis: 20% of COVID-19 research papers have more than 8 listed authors and this portion of the dataset accounts for over 60 of the 100 most cited publications relating to the coronavirus. Yet, the majority of COVID-19 research papers (53%) have 4 or fewer authors. We find no evidence that the reduced Collaboration Index of COVID-19 research is due to working conditions and restrictions during the pandemic. Despite a global shift towards remote work, research in 2020 continues the recent trend of increasing collaboration. The preference for smaller research teams appears to be specific to COVID-19 research and not simply a factor of research during COVID-19.

The prevalence of smaller research teams is important to understand about COVID-19 research. Smaller teams have been shown to play a different role to larger teams in both research and technology [wu_2019]. In an analysis of research collaborations, Wu et al. show that small research teams can disrupt science and technology by exploring and amplifying promising ideas from older and less-popular work, while large teams develop on recent successes by solving acknowledged problems [wu_2019]. The definition by Wu et al. of disruptive articles relates closely to the metric of betweenness centrality for citation networks. That is, disruptive papers can connect otherwise separate communities in a research network. We find some evidence that COVID-19 research may be increasing the connectivity between disciplines, as authors are more likely to publish across multiple fields and research teams are more diverse. A trend towards greater levels of multidisciplinary collaboration has been identified in many scientific disciplines [porter_2008]. This trend is evident in the non-COVID-19 portions of our dataset. Research teams of fewer than 10 members publishing in 2020 exhibit greater disciplinary diversity than similarly-sized teams publishing in 2019, for example. Likewise, the number of authors publishing in multiple disciplines is increasing steadily year-on-year. In COVID-19 research, the increase in multidisciplinarity (of both teams and individuals) exceeds the established trend. This may be evidence of the disruptive nature of COVID-related research. Below, we use flow diagrams to explore author multidisciplinarity in specific topics in the COVID-related research dataset.

Figures 4, 5, 6 and 7 present four selected case studies of author multidisciplinarity in COVID-related research in 2020. To provide a clear visualisation of the strongest trends that exist, each FoS network shows only the 50 edges with the greatest weights. We choose Virology as a case study because it is largest subset in COVID-related research, while Computer Science and Materials Science were chosen to show considerable increases in author multidisciplinarity in 2020 (see figure 2), and Development Economics presents with a very diverse set of contributing disciplines. For example, Figure 4 shows the intersection between Medicine, Biology and Chemistry in COVID-19 research relating to Virology. Sub-disciplines Molecular Biology, Biochemistry, Immunology, and Virology all appear closely related in this graph. They are strongly interconnected, indicating many instances of authors publishing between disciplines and each acts as both a source and as a destination in the network, as authors who publish in any of these sub-disciplines prior to COVID-19 are likely to publish in the others during COVID-19. Figure 5 illustrates the multidisciplinary nature of Computer Science research in COVID-19. Unlike the Virology graph in figure 4, there are only two destinations in this network: Computer Science and Medicine. Computer Science research in the COVID-19 dataset is primarily focused on Machine Learning solutions to automating COVID-19 detection from medical images [nguyen_2021] (See Supplementary Table 6(a)). This effort is evident in the graph, as Computer Science research in COVID-19 is most commonly characterised within the sub-disciplines Machine Learning, Artificial Intelligence, Pathology, Surgery and Algorithm. Also evident is the multidisciplinary nature of the effort, as researchers with backgrounds in many of the S.T.E.M. fields are shown to contribute. Figure 6 reports the FoS network for COVID-19 research relating to Materials Science. The graphs illustrates an intersection between the fields of Physics, Chemistry, Engineering and Materials Science as researchers from each of these disciplines contributes to coronavirus research. Many of the most cited articles in this subset relate to airborne particles and the efficacy of face masks [lustig_2020], along with the use of electrochemical biosensors for pathogen detection [cesewski_2020] (See Supplementary Table 7(a)). Figure 7 presents the FoS network for the COVID-19 related research papers in the field of Development Economics. Some of the most cited articles in this subset concern studies of the socio-economic implications and effects of the pandemic globally [nicola_2020, walker_2020], and of health inequity in low- and middle-income countries [patel_2020, wang_2020_equity] (See Supplementary Table 8(a)). Research in this subset is characterised by the diverse set of sub-disciplines shown on the left of the figure, as authors with backgrounds in social science, social psychology, medicine, statistics, economics, and biology are all found to contribute.

The methods outlined in this work could be applied in future scientometric analyses to assess and visualise multidisciplinarity in a body of research. This may be of interest to researchers seeking to understand the evolution of their own field of study, or to funding agencies who recognise the established benefits of multidisciplinary collaboration. In the case of this work, we show COVID-19 research teams to be smaller yet more multidisciplinary than non-COVID-19 teams. It is suggested in early work that authors publishing COVID-19 research favoured smaller, less international collaborations in order to reduce co-ordination costs and contribute to the public health effort sooner [fry_2020]. We would like to elaborate on this characterisation of collaboration in COVID-19 research; adding that authors sought to minimise the limitations of working in smaller teams by collaborating with scientists from diverse research backgrounds. That is to say, in the urgency of the pandemic, scientists favour smaller, more multidisciplinary research teams in order to collaborate more efficiently.

Figure 4: Author Multidisciplinarity in Virology Research in COVID-19. The graph relates an author’s research background to the fields they publish in COVID-related articles. This network is produced from 22,561 COVID-related research papers which were assigned the MAG field ‘Virology’. Pre-COVID sub-disciplines (common in the research backgrounds of the authors) are shown on the left and COVID-related sub-disciplines (common in the article subset) are shown on the right. Sub-disciplines are coloured by their parent disciplines and edges are assigned the colour of the pre-2020 node. Edges are weighted by the numbers of authors who published in both of the corresponding sub-disciplines. The bi-gram terms which occurred most frequently in the titles of these papers were: COVID-19 pandemic, coronavirus disease, SARS-CoV-2 infection and novel coronavirus. (See Supplementary Table 5).
Figure 5: Author Multidisciplinarity in Computer Science Research in COVID-19. This network is produced from 9,004 COVID-related research papers which were attributed the MAG field ‘Computer Science’. The bi-gram terms which occurred most frequently in the titles of these papers were:

COVID-19 pandemic, deep learning, neural network, machine learning, contact tracing

and chest x-ray. *The MAG sub-discipline ‘Algorithm’ is a level 1 parent for any algorithms identified in the fields of study. The most frequently occurring children of the Algorithm field in this subset are

‘artificial neural network’, ‘cluster analysis’, ‘inference’,

and

support vector machine

. (See Supplementary Table 6).
Figure 6: Author Multidisciplinarity in Materials Science Research in COVID-19. This network is produced from 1,229 COVID-related research papers which were attributed the MAG field ‘Materials Science’. The bi-gram terms which occurred most frequently in the titles of these papers were: filtration efficiency, additive manufacturing, and face mask. (See Supplementary Table 7).
Figure 7: Author Multidisciplinarity in Development Economics Research in COVID-19. The graph relates an author’s research background to the fields they publish in COVID-related articles. This network is produced from 1,564 COVID-related research papers which were attributed the MAG field ‘Development Economics’. The most cited articles in this subset concern studies of the socio-economic implications and effects of the pandemic globally [nicola_2020, walker_2020], and of health inequity in low- and middle-income countries [patel_2020, wang_2020_equity]. (See Supplementary Table 8).

References

Additional information

Acknowledgments

This research was supported by Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289_P2.

Data availability

The data used in our study can reproduced from the set of Microsoft Academic Graph article IDs, which will be made available upon request.