Manas: Mining Software Repositories to Assist AutoML

by   Giang Nguyen, et al.

Today deep learning is widely used for building software. A software engineering problem with deep learning is that finding an appropriate convolutional neural network (CNN) model for the task can be a challenge for developers. Recent work on AutoML, more precisely neural architecture search (NAS), embodied by tools like Auto-Keras aims to solve this problem by essentially viewing it as a search problem where the starting point is a default CNN model, and mutation of this CNN model allows exploration of the space of CNN models to find a CNN model that will work best for the problem. These works have had significant success in producing high-accuracy CNN models. There are two problems, however. First, NAS can be very costly, often taking several hours to complete. Second, CNN models produced by NAS can be very complex that makes it harder to understand them and costlier to train them. We propose a novel approach for NAS, where instead of starting from a default CNN model, the initial model is selected from a repository of models extracted from GitHub. The intuition being that developers solving a similar problem may have developed a better starting point compared to the default model. We also analyze common layer patterns of CNN models in the wild to understand changes that the developers make to improve their models. Our approach uses commonly occurring changes as mutation operators in NAS. We have extended Auto-Keras to implement our approach. Our evaluation using 8 top voted problems from Kaggle for tasks including image classification and image regression shows that given the same search time, without loss of accuracy, Manas produces models with 42.9 on GPU, Manas' models train 30.3


page 1

page 2

page 3

page 4


Single-Path NAS: Device-Aware Efficient ConvNet Design

Can we automatically design a Convolutional Network (ConvNet) with the h...

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been d...

Mutation is all you need

Neural architecture search (NAS) promises to make deep learning accessib...

Shortest Edit Path Crossover: A Theory-driven Solution to the Permutation Problem in Evolutionary Neural Architecture Search

Evolutionary algorithms (EAs) have gained attention recently due to thei...

Evaluating Generic Auto-ML Tools for Computational Pathology

Image analysis tasks in computational pathology are commonly solved usin...

Colab NAS: Obtaining lightweight task-specific convolutional neural networks following Occam's razor

The current trend of applying transfer learning from CNNs trained on lar...

Mining Fix Patterns for FindBugs Violations

In this paper, we first collect and track large-scale fixed and unfixed ...

Please sign up or login with your details

Forgot password? Click here to reset