Single-shot Embedding Dimension Search in Recommender System

04/07/2022
by   Liang Qu, et al.
0

As a crucial component of most modern deep recommender systems, feature embedding maps high-dimensional sparse user/item features into low-dimensional dense embeddings. However, these embeddings are usually assigned a unified dimension, which suffers from the following issues: (1) high memory usage and computation cost. (2) sub-optimal performance due to inferior dimension assignments. In order to alleviate the above issues, some works focus on automated embedding dimension search by formulating it as hyper-parameter optimization or embedding pruning problems. However, they either require well-designed search space for hyperparameters or need time-consuming optimization procedures. In this paper, we propose a Single-Shot Embedding Dimension Search method, called SSEDS, which can efficiently assign dimensions for each feature field via a single-shot embedding pruning operation while maintaining the recommendation accuracy of the model. Specifically, it introduces a criterion for identifying the importance of each embedding dimension for each feature field. As a result, SSEDS could automatically obtain mixed-dimensional embeddings by explicitly reducing redundant embedding dimensions based on the corresponding dimension importance ranking and the predefined parameter budget. Furthermore, the proposed SSEDS is model-agnostic, meaning that it could be integrated into different base recommendation models. The extensive offline experiments are conducted on two widely used public datasets for CTR prediction tasks, and the results demonstrate that SSEDS can still achieve strong recommendation performance even if it has reduced 90% parameters. Moreover, SSEDS has also been deployed on the WeChat Subscription platform for practical recommendation services. The 7-day online A/B test results show that SSEDS can significantly improve the performance of the online recommendation model.

READ FULL TEXT
research
08/09/2022

OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction

Learning embedding table plays a fundamental role in Click-through rate(...
research
06/08/2020

Differentiable Neural Input Search for Recommender Systems

Latent factor models are the driving forces of the state-of-the-art reco...
research
01/19/2021

Learnable Embedding Sizes for Recommender Systems

The embedding-based representation learning is commonly used in deep lea...
research
04/01/2022

i-Razor: A Neural Input Razor for Feature Selection and Dimension Search in Large-Scale Recommender Systems

Input features play a crucial role in the predictive performance of DNN-...
research
08/17/2022

Field-wise Embedding Size Search via Structural Hard Auxiliary Mask Pruning for Click-Through Rate Prediction

Feature embeddings are one of the most essential steps when training dee...
research
06/04/2021

Learning Elastic Embeddings for Customizing On-Device Recommenders

In today's context, deploying data-driven services like recommendation o...
research
11/05/2021

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

Feature embedding learning and feature interaction modeling are two cruc...

Please sign up or login with your details

Forgot password? Click here to reset