AutoDis: Automatic Discretization for Embedding Numerical Features in CTR Prediction

by   Huifeng Guo, et al.

Learning sophisticated feature interactions is crucial for Click-Through Rate (CTR) prediction in recommender systems. Various deep CTR models follow an Embedding Feature Interaction paradigm. The majority focus on designing network architectures in Feature Interaction module to better model feature interactions while the Embedding module, serving as a bottleneck between data and Feature Interaction module, has been overlooked. The common methods for numerical feature embedding are Normalization and Discretization. The former shares a single embedding for intra-field features and the latter transforms the features into categorical form through various discretization approaches. However, the first approach surfers from low capacity and the second one limits performance as well because the discretization rule cannot be optimized with the ultimate goal of CTR model. To fill the gap of representing numerical features, in this paper, we propose AutoDis, a framework that discretizes features in numerical fields automatically and is optimized with CTR models in an end-to-end manner. Specifically, we introduce a set of meta-embeddings for each numerical field to model the relationship among the intra-field features and propose an automatic differentiable discretization and aggregation approach to capture the correlations between the numerical features and meta-embeddings. Comprehensive experiments on two public and one industrial datasets are conducted to validate the effectiveness of AutoDis over the SOTA methods.


page 1

page 2

page 3

page 4


i-Razor: A Neural Input Razor for Feature Selection and Dimension Search in Large-Scale Recommender Systems

Input features play a crucial role in the predictive performance of DNN-...

Feature embedding in click-through rate prediction

We tackle the challenge of feature embedding for the purposes of improvi...

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

Feature embedding learning and feature interaction modeling are two cruc...

FLEN: Leveraging Field for Scalable CTR Prediction

Click-Through Rate (CTR) prediction has been an indispensable component ...

Memory-efficient Embedding for Recommendations

Practical large-scale recommender systems usually contain thousands of f...

On Embeddings for Numerical Features in Tabular Deep Learning

Recently, Transformer-like deep architectures have shown strong performa...

AEFE: Automatic Embedded Feature Engineering for Categorical Features

The challenge of solving data mining problems in e-commerce applications...