DeepAI AI Chat
Log In Sign Up

Repo2Vec: A Comprehensive Embedding Approach for Determining Repository Similarity

by   Md Omar Faruk Rokon, et al.
University of California, Riverside

How can we identify similar repositories and clusters among a large online archive, such as GitHub? Determiningrepository similarity is an essential building block in studying the dynamics and the evolution of such software ecosystems. The key challenge is to determine the right representation for the diverse repository features in a way that: (a) it captures all aspects of the available information, and (b) it is readily usable by MLalgorithms. We propose Repo2Vec, a comprehensive embedding approach to represent a repository as a distributed vector by combining features from three types of information sources. As our key novelty, we consider three types of information: (a)metadata, (b) the structure of the repository, and (c) the source code. We also introduce a series of embedding approaches to represent and combine these information types into a single embedding. We evaluate our method with two real datasets from GitHub for a combined 1013 repositories. First, we show that our method outperforms previous methods in terms of precision (93 nearly twice as many Strongly Similar repositories and 30 Positives. Second, we show how Repo2Vecprovides a solid basis for: (a) distinguishing between malware and benign repositories, and (b) identifying a meaningful hierarchical clustering. For example, we achieve 98 96 is a fundamental building block for enabling many repository analysis functions such as repository categorization by target platform or intention, detecting code-reuse and clones, and identifying lineage and evolution.


page 1

page 4

page 5

page 7

page 8

page 9


SourceFinder: Finding Malware Source-Code from Publicly Available Repositories

Where can we find malware source code? This question is motivated by a r...

Topical: Learning Repository Embeddings from Source Code using Attention

Machine learning on source code (MLOnCode) promises to transform how sof...

Scalable Program Clone Search Through Spectral Analysis

We consider the problem of program clone search, i.e. given a target pro...

Bug-Fix Variants: Visualizing Unique Source Code Changes across GitHub Forks

Forking is a common practice for developers when building upon on alread...

OWL Reasoners still useable in 2023

In a systematic literature and software review over 100 OWL reasoners/sy...

Weaponizing Unicodes with Deep Learning – Identifying Homoglyphs with Weakly Labeled Data

Visually similar characters, or homoglyphs, can be used to perform socia...