Learning Elastic Embeddings for Customizing On-Device Recommenders

06/04/2021
by   Tong Chen, et al.
0

In today's context, deploying data-driven services like recommendation on edge devices instead of cloud servers becomes increasingly attractive due to privacy and network latency concerns. A common practice in building compact on-device recommender systems is to compress their embeddings which are normally the cause of excessive parameterization. However, despite the vast variety of devices and their associated memory constraints, existing memory-efficient recommender systems are only specialized for a fixed memory budget in every design and training life cycle, where a new model has to be retrained to obtain the optimal performance while adapting to a smaller/larger memory budget. In this paper, we present a novel lightweight recommendation paradigm that allows a well-trained recommender to be customized for arbitrary device-specific memory constraints without retraining. The core idea is to compose elastic embeddings for each item, where an elastic embedding is the concatenation of a set of embedding blocks that are carefully chosen by an automated search function. Correspondingly, we propose an innovative approach, namely recommendation with universally learned elastic embeddings (RULE). To ensure the expressiveness of all candidate embedding blocks, RULE enforces a diversity-driven regularization when learning different embedding blocks. Then, a performance estimator-based evolutionary search function is designed, allowing for efficient specialization of elastic embeddings under any memory constraint for on-device recommendation. Extensive experiments on real-world datasets reveal the superior performance of RULE under tight memory budgets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/18/2023

Personalized Elastic Embedding Learning for On-Device Recommendation

To address privacy concerns and reduce network latency, there has been a...
research
04/07/2023

Continuous Input Embedding Size Search For Recommender Systems

Latent factor models are the most popular backbones for today's recommen...
research
09/07/2023

Learning Compact Compositional Embeddings via Regularized Pruning for Recommendation

Latent factor models are the dominant backbones of contemporary recommen...
research
09/27/2022

Efficient On-Device Session-Based Recommendation

On-device session-based recommendation systems have been achieving incre...
research
04/07/2022

Single-shot Embedding Dimension Search in Recommender System

As a crucial component of most modern deep recommender systems, feature ...
research
04/23/2022

On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation

Modern recommender systems operate in a fully server-based fashion. To c...
research
08/14/2023

AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation

In the domain of streaming recommender systems, conventional methods for...

Please sign up or login with your details

Forgot password? Click here to reset