When Deep Learning Met Code Search

by   Jose Cambronero, et al.

There have been multiple recent proposals on using deep neural networks for code search using natural language. Common across these proposals is the idea of embedding code and natural language queries, into real vectors and then using vector distance to approximate semantic correlation between code and the query. Multiple approaches exist for learning these embeddings, including unsupervised techniques, which rely only on a corpus of code examples, and supervised techniques, which use an aligned corpus of paired code and natural language descriptions. The goal of this supervision is to produce embeddings that are more similar for a query and the corresponding desired code snippet. Clearly, there are choices in whether to use supervised techniques at all, and if one does, what sort of network and training to use for supervision. This paper is the first to evaluate these choices systematically. To this end, we assembled implementations of state-of-the-art techniques to run on a common platform, training and evaluation corpora. To explore the design space in network complexity, we also introduced a new design point that is a minimal supervision extension to an existing unsupervised technique. Our evaluation shows that: 1. adding supervision to an existing unsupervised technique can improve performance, though not necessarily by much; 2. simple networks for supervision can be more effective that more sophisticated sequence-based networks for code search; 3. while it is common to use docstrings to carry out supervision, there is a sizeable gap between the effectiveness of docstrings and a more query-appropriate supervision corpus.


Neural Code Search Evaluation Dataset

There has been an increase of interest in code search using natural lang...

Code Search based on Context-aware Code Translation

Code search is a widely used technique by developers during software dev...

COSEA: Convolutional Code Search with Layer-wise Attention

Semantic code search, which aims to retrieve code snippets relevant to a...

A Coherent Unsupervised Model for Toponym Resolution

Toponym Resolution, the task of assigning a location mention in a docume...

Beryllium: Neural Search for Algorithm Implementations

In this paper, we explore the feasibility of finding algorithm implement...

Evaluation of Siamese Networks for Semantic Code Search

With the increase in the number of open repositories and discussion foru...

Dynamic Boundary Time Warping for Sub-sequence Matching with Few Examples

The paper presents a novel method of finding a fragment in a long tempor...

Please sign up or login with your details

Forgot password? Click here to reset