Differentiable Neural Input Search for Recommender Systems

06/08/2020
by   Weiyu Cheng, et al.
0

Latent factor models are the driving forces of the state-of-the-art recommender systems, with an important insight of vectorizing raw input features into dense embeddings. The dimensions of different feature embeddings are often set to a uniform value manually or through grid search, which may yield suboptimal model performance. Existing work applied heuristic methods or reinforcement learning to search for varying embedding dimensions. However, the embedding dimension per feature is rigidly chosen from a restricted set of candidates due to the scalability issue involved in the optimization process over a large search space. In this paper, we propose a differentiable neural input search algorithm towards learning more flexible dimensions of feature embeddings, namely a mixed dimension scheme, leading to better recommendation performance and lower memory cost. Our method can be seamlessly incorporated with various existing architectures of latent factor models for recommendation. We conduct experiments with 6 state-of-the-art model architectures on two typical recommendation tasks: Collaborative Filtering (CF) and Click-Through-Rate (CTR) prediction. The results demonstrate that our method achieves the best recommendation performance compared with 3 neural input search approaches over all the model architectures, and can reduce the number of embedding parameters by 2x and 20x on CF and CTR prediction, respectively.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/07/2022

Single-shot Embedding Dimension Search in Recommender System

As a crucial component of most modern deep recommender systems, feature ...
research
04/07/2023

Continuous Input Embedding Size Search For Recommender Systems

Latent factor models are the most popular backbones for today's recommen...
research
07/10/2019

Neural Input Search for Large Scale Recommendation Models

Recommendation problems with large numbers of discrete items, such as pr...
research
09/14/2023

iHAS: Instance-wise Hierarchical Architecture Search for Deep Learning Recommendation Models

Current recommender systems employ large-sized embedding tables with uni...
research
06/26/2020

Memory-efficient Embedding for Recommendations

Practical large-scale recommender systems usually contain thousands of f...
research
08/09/2022

OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction

Learning embedding table plays a fundamental role in Click-through rate(...
research
09/25/2019

Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems

In many real-world applications, e.g. recommendation systems, certain it...

Please sign up or login with your details

Forgot password? Click here to reset