Efficient Automatic Machine Learning via Design Graphs

10/21/2022
by   Shirley Wu, et al.
1

Despite the success of automated machine learning (AutoML), which aims to find the best design, including the architecture of deep networks and hyper-parameters, conventional AutoML methods are computationally expensive and hardly provide insights into the relations of different model design choices. To tackle the challenges, we propose FALCON, an efficient sample-based method to search for the optimal model design. Our key insight is to model the design space of possible model designs as a design graph, where the nodes represent design choices, and the edges denote design similarities. FALCON features 1) a task-agnostic module, which performs message passing on the design graph via a Graph Neural Network (GNN), and 2) a task-specific module, which conducts label propagation of the known model performance information on the design graph. Both modules are combined to predict the design performances in the design space, navigating the search direction. We conduct extensive experiments on 27 node and graph classification tasks from various application domains, and an image classification task on the CIFAR-10 dataset. We empirically show that FALCON can efficiently obtain the well-performing designs for each task using only 30 explored nodes. Specifically, FALCON has a comparable time cost with the one-shot approaches while achieving an average improvement of 3.3 with the best baselines.

READ FULL TEXT
research
09/17/2022

Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach

Graph Neural Networks (GNNs) are popular machine learning methods for mo...
research
03/15/2022

Incorporating Heterophily into Graph Neural Networks for Graph Classification

Graph neural networks (GNNs) often assume strong homophily in graphs, se...
research
03/14/2023

AutoTransfer: AutoML with Knowledge Transfer – An Application to Graph Neural Networks

AutoML has demonstrated remarkable success in finding an effective neura...
research
11/17/2020

Design Space for Graph Neural Networks

The rapid evolution of Graph Neural Networks (GNNs) has led to a growing...
research
06/14/2023

NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification

Graph neural networks have been extensively studied for learning with in...
research
01/09/2023

SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks

Understanding traffic scenes requires considering heterogeneous informat...
research
06/12/2019

Neural Graph Evolution: Towards Efficient Automatic Robot Design

Despite the recent successes in robotic locomotion control, the design o...

Please sign up or login with your details

Forgot password? Click here to reset