A Dataset and an Approach for Identity Resolution of 38 Million Author IDs extracted from 2B Git Commits

by   Tanner Fry, et al.

The data collected from open source projects provide means to model large software ecosystems, but often suffer from data quality issues, specifically, multiple author identification strings in code commits might actually be associated with one developer. While many methods have been proposed for addressing this problem, they are either heuristics requiring manual tweaking, or require too much calculation time to do pairwise comparisons for 38M author IDs in, for example, the World of Code collection. In this paper, we propose a method that finds all author IDs belonging to a single developer in this entire dataset, and share the list of all author IDs that were found to have aliases. To do this, we first create blocks of potentially connected author IDs and then use a machine learning model to predict which of these potentially related IDs belong to the same developer. We processed around 38 million author IDs and found around 14.8 million IDs to have an alias, which belong to 5.4 million different developers, with the median number of aliases being 2 per developer. This dataset can be used to create more accurate models of developer behaviour at the entire OSS ecosystem level and can be used to provide a service to rapidly resolve new author IDs.


page 1

page 2

page 3

page 4


Deep Author Name Disambiguation using DBLP Data

In the academic world, the number of scientists grows every year and so ...

ALFAA: Active Learning Fingerprint Based Anti-Aliasing for Correcting Developer Identity Errors in Version Control Data

Graphs of developer networks are important for software engineering rese...

Geographic Diversity in Public Code Contributions

We conduct an exploratory, large-scale, longitudinal study of 50 years o...

ICodeNet – A Hierarchical Neural Network Approach for Source Code Author Identification

With the open-source revolution, source codes are now more easily access...

Statistical mechanics of neocortical interactions: Portfolio of Physiological Indicators

There are several kinds of non-invasive imaging methods that are used to...

Scaling Author Name Disambiguation with CNF Blocking

An author name disambiguation (AND) algorithm identifies a unique author...

A large dataset of software mentions in the biomedical literature

We describe the CZ Software Mentions dataset, a new dataset of software ...

Please sign up or login with your details

Forgot password? Click here to reset