We employ logistic regression to test the relationship between gender and Wikipedia recognition, controlling for h-index. For details of data collection and gender detection see Materials and Methods. We start with a simple model that has the following structure
where denotes the existence of a Wikipedia page, denotes the gender of the scientist, their h-index, and and are the model constants. This model assumes that being male or female changes the chance of recognition irrespective of academic achievement. In the nest step, considering that scientific accomplishments by females might be viewed differently, we add a new term to the model with an interaction between h-index and gender,
|Dependent variable: Wikipedia page exists|
|Akaike Inf. Crit.||12,777.900||12,136.920||12,109.210|
|Note:||p0.1; p0.05; p0.01|
The results from the fit of the model to the data presented in Table 2 point towards structural discrimination in the recognition of scientific achievement. Regardless of field of study, being male significantly increases the chance of being recognised and featured on Wikipedia.
The negative interaction effect between gender and h-index suggests that Wikipedia’s bias towards men is strongest amongst scientists with relatively low indexes. Gender plays a smaller role in the recognition of academics with exceptional academic standing.
Logistic regression produces log-odds as coefficients; using those, we have plotted the probability that an economist of both genders is recognised with a Wikipedia page at different h-index levels (Figure1). A female economist with an average h-index has a probability of 0.11 of being recognised by Wikipedia, while an average male economist has a probability of 0.18. A male economist has to achieve an h-index of 11 for a similar probability of public recognition as a female economist with an h-index of 19. Similar patterns are observed for Physics and Philosophy. Women are 19%, 37% and 50% less likely to receive recognition than male peers when both have an average h-index in Physics, Economics and Philosophy respectively. We calculate these percentages by dividing the predicted probability of a women with an average h-index of having a Wikipedia page by the predicted probability for a man with the same h-index to have a Wikipedia page in the same field of research.
To check the robustness, we provide a number of variations of this model to see if the effects hold. To control for cross-discipline differences, we run separate models per field to investigate the differences between fields (see Table S1). To further check the robustness of the results, we use alternative measures for scientific achievement such as raw number of citations and h5 index to test if that changes the outcomes (see Table S2). This finding holds when controlling for field (see Table 2), when run separately for every field (see Table S1) and when alternative measures are used (see Table S2). It is statistically significant at in all analyses.
We report on evidence of a bias against recognising the scientific accomplishments of women on Wikipedia. Men are more likely to be awarded a page in the world’s most influential encyclopedia than women with similar scientometric records. This finding is replicated in Physics (natural sciences), Economics (social sciences), and Philosophy (humanities). The magnitude of male advantage is remarkably similar across the disparate fields.
It is beyond the scope of this paper to establish the causal mechanism behind the gender gap in recognition. Is research from women taken less seriously? Are males more easily given access to public fora to discuss their findings? And one should note that the biases reported in this work are on top of the reported biases on research funding allocations (21), publishing practices and hiring exercises (22, 23).
We must also note that a portion of the reported bias might be due to the known gender gap among Wikipedia editors. It is notable that there are few female editors amongst the ranks of Wikipedia editors (17, 24). The Wikimedia Foundation might want to consider policy changes to give women equal recognition for equal work as an starting point to battle this societal malfunction in a wider scope.
Materials and Methods
The analysis is conducted with three measures: scientific accomplishment (retrieved with Google Scholar), gender (retrieved from genderize.io) and recognition from Wikipedia (retrieved from the Wikipedia API). We will cover each measure in the following sections. The summary statistics are available in Table 1 and in Table S3.
0.1 Scientific Accomplishment
While scientists receive many forms of recognition, the most common measure is the citation. Citation metrics have increased in importance in the scientific realm. The h-index is widely preferred over raw citation counts, because it accounts for both the number of publications and citations (25). Many universities set minimum h-index values for new hires, and some universities base promotions on h-index thresholds (26).
The source of our dataset is Google Scholar. We queried a particular field and collected names in the order Google Scholar presents them, which is ordered by citation count. For every scientist, we retrieved citation counts, their name and their institution. Google Scholar has been found to have the largest coverage as compared to other databases, with up to 33% more authors than its direct competitors and more diverse publications, such as conference papers and books (27, 28, 29). Thus, we are satisfied that Google Scholar gives an accurate and comprehensive overview of active scholars and their citations.
Collecting data from Google Scholar is laborious, so we sampled scholars from three fields in different parts of the academic world: Physics (natural sciences), Economics (social sciences) and Philosophy (humanities) (30). As reported in Table 1, the number of scholars in our sample, the gender balance and proportion of scholars with Wikipedia pages differs per field.
Our sample of scholars is non-random, because scientists were ordered by h-index. However, we collected the top 10,000 available scholars from a field. The ‘bottom’ of our sample contains scholars with h-indices as low as 1, so we cover a wide range of achievement. If we missed scholars, they must have very few citations and publications. This does not compromise our analysis, because these scholars are not likely to receive recognition from Wikipedia and thus not relevant. All three citation measures are not normally distributed (See Figures S1-S3) and transformed for the regression analysis.
Google Scholar does not list the gender of a scientist. Therefore, we must detect the gender of a scholar based on their first name. This technique is widely used and accurate (2, 31). We use genderize.io API, which makes use of a database of 216286 names from 79 countries and 89 languages to make prediction. Conveniently, genderize.io reports the number of times a given name appears in their database and the proportion of the two sexes. We applied strict filters: only predictions with a confidence greater than 90% based on a minimum sample size of 10 were accepted. This measure cut our sample down to 15,049 from 23.000 scholars collected via Google Scholar.
Genderize makes the assumption that persons who are a woman identify as female. However, both sexes can identify as many genders. The analysis would be superior if we could use the identified genders of every scientist, but this possibility is not available. Given that it is common for women to identify as female and men as male, we use the Genderize categorization as the closest available proxy.
0.3 Recognition by Wikipedia
We queried the Wikipedia API with the names of scholars to check for Wikipedia pages under their name. When the Wikipedia page is listed under a slightly different name or a known alias, the Wikipedia API automatically refers us to the correct page. We checked a sample of 30 codings manually and found no miscodings.
We thank Jop Flameling for discussion on the research design and data collection. TY was partially supported by the Alan Turing Institute under the EPSRC grant no. EP/N510129/1.
- (1) Raymond J (2013) Most of us are biased. Nature 495(7439):33–34.
- (2) Larivière V, Ni C, Gingras Y, Cronin B, Sugimoto CR (2013) Bibliometrics: Global gender disparities in science. Nature 504(7479):211–213.
- (3) Clancy KBH, Nelson RG, Rutherford JN, Hinde K (2014) Survey of Academic Field Experiences (SAFE): Trainees Report Harassment and Assault. PLoS ONE 9(7):e102172.
- (4) Bornmann L, Mutz R, Daniel HD (2007) Gender differences in grant peer review: A meta-analysis. Journal of Informetrics 1(3):226–238.
- (5) Boyle PJ, Smith LK, Cooper NJ, Williams KS, O’Connor H (2015) Gender balance: Women are funded more fairly in social science. Nature 525(7568):181–183.
- (6) Nosek BA, et al. (2009) National differences in gender-science stereotypes predict national sex differences in science and math achievement. Proceedings of the National Academy of Sciences 106(26):10593–10597.
- (7) Pollack E (2013) Why Are There Still So Few Women in Science?
- (8) Yang Y, Chawla NV, Uzzi B (2019) A network’s gender composition and communication pattern predict women’s leadership success. Proceedings of the National Academy of Sciences 116(6):2033–2038.
- (9) Melville S, Eccles K, Yasseri T (2019) Topic modeling of everyday sexism project entries. Frontiers in Digital Humanities 5:28.
- (10) Giles J (2005) Internet encyclopaedias go head to head.
- (11) Halavais A, Lackaff D (2008) An analysis of topical coverage of wikipedia. Journal of computer-mediated communication 13(2):429–440.
- (12) Greenstein S, Zhu F (2012) Is wikipedia biased? American Economic Review 102(3):343–48.
Samoilenko A, Yasseri T (2014) The distorted mirror of wikipedia: a
quantitative analysis of wikipedia coverage of academics.
EPJ data science3(1):1.
- (14) García-Gavilanes R, Tsvetkova M, Yasseri T (2016) Dynamics and biases of online attention: the case of aircraft crashes. Royal Society Open Science 3(10):160460.
- (15) García-Gavilanes R, Mollgaard A, Tsvetkova M, Yasseri T (2017) The memory remains: Understanding collective memory in the digital age. Science advances 3(4):e1602368.
- (16) Devlin H (2018) Academic writes 270 Wikipedia pages in a year to get female scientists noticed.
- (17) Wade J, Zaringhalam M (2018) Why we’re editing women scientists onto Wikipedia. Nature.
- (18) Reagle J, Rhue L (2011) Gender bias in wikipedia and britannica. International Journal of Communication 5:21.
Halfaker A (2017) Interpolating quality dynamics in wikipedia and demonstrating the keilana effect inProceedings of the 13th International Symposium on Open Collaboration. (ACM), p. 19.
- (20) Graells-Garrido E, Lalmas M, Menczer F (2015) First women, second sex: Gender bias in wikipedia in Proceedings of the 26th ACM Conference on Hypertext & Social Media. (ACM), pp. 165–174.
- (21) Head MG, Fitchett JR, Cooke MK, Wurie FB, Atun R (2013) Differences in research funding for women scientists: a systematic comparison of uk investments in global infectious disease research during 1997–2010. BMJ open 3(12):e003362.
- (22) Carr PL, Szalacha L, Barnett R, Caswell C, Inui T (2003) A" ton of feathers": gender discrimination in academic medical careers and how to manage it. Journal of Women’s Health 12(10):1009–1018.
- (23) Clauset A, Arbesman S, Larremore DB (2015) Systematic inequality and hierarchy in faculty hiring networks. Science advances 1(1):e1400005.
Hill BM, Shaw A (2013) The wikipedia gender gap revisited: Characterizing survey response bias with propensity score estimation.PloS one 8(6):e65782.
- (25) Kelly CD, Jennions MD (2006) The h index and career assessment by numbers. Trends in Ecology & Evolution 21(4):167–170.
- (26) Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I (2015) Bibliometrics: The Leiden Manifesto for research metrics. Nature 520(7548):429–431.
- (27) Meho LI, Yang K (2007) Impact of data sources on citation counts and rankings of lis faculty: Web of science versus scopus and google scholar. Journal of the american society for information science and technology 58(13):2105–2125.
- (28) Kousha K, Thelwall M, Rezaie S (2011) Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology 62(11):2147–2164.
Harzing AW (2013) A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners.Scientometrics 94(3):1057–1075.
- (30) Holstege F (2019) Googlescholar_research (https://github.com/fholstege/GoogleScholar_Research).
- (31) Karimi F, Wagner C, Lemmerich F, Jadidi M, Strohmaier M (2016) Inferring gender from names on the web: A comparative evaluation of gender detection methods in Proceedings of the 25th International Conference Companion on World Wide Web. (International World Wide Web Conferences Steering Committee), pp. 53–54.
|Akaike Inf. Crit.||4,673.403||5,545.226||1,892.215|
|Note:||p0.1; p0.05; p0.01|
|Akaike Inf. Crit.||12,852.940||12,676.030|
|Note:||p0.1; p0.05; p0.01|