NASRec: Weight Sharing Neural Architecture Search for Recommender Systems

07/14/2022
by   Tunhou Zhang, et al.
0

The rise of deep neural networks provides an important driver in optimizing recommender systems. However, the success of recommender systems lies in delicate architecture fabrication, and thus calls for Neural Architecture Search (NAS) to further improve its modeling. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures, with the supernet incorporating versatile operator choices and dense connectivity minimizing human prior for flexibility. The scale and heterogeneity in NASRec impose challenges in search, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our results on three Click-Through Rates (CTR) prediction benchmarks show that NASRec can outperform both manually designed models and existing NAS methods, achieving state-of-the-art performance.

READ FULL TEXT
research
03/03/2020

ADWPNAS: Architecture-Driven Weight Prediction for Neural Architecture Search

How to discover and evaluate the true strength of models quickly and acc...
research
10/14/2022

AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers

Neural architecture search (NAS) has demonstrated promising results on i...
research
06/10/2020

AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System

User behavior and feature interactions are crucial in deep learning-base...
research
06/29/2020

Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction

Click-Through Rate (CTR) prediction is one of the most important machine...
research
04/08/2022

SuperNet in Neural Architecture Search: A Taxonomic Survey

Deep Neural Networks (DNN) have made significant progress in a wide rang...
research
08/29/2021

Analyzing and Mitigating Interference in Neural Architecture Search

Weight sharing has become the de facto approach to reduce the training c...
research
11/28/2022

PIDS: Joint Point Interaction-Dimension Search for 3D Point Cloud

The interaction and dimension of points are two important axes in design...

Please sign up or login with your details

Forgot password? Click here to reset