Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction

by   Yanqiao Zhu, et al.

Click-through rate (CTR) prediction, which aims to predict the probability that whether of a user will click on an item, is an essential task for many online applications. Due to the nature of data sparsity and high dimensionality in CTR prediction, a key to making effective prediction is to model high-order feature interactions among feature fields. To explicitly model high-order feature interactions, an efficient way is to stack multihead self-attentive neural networks, which has achieved promising performance. However, one problem of the vanilla self-attentive network is that two terms, a whitened pairwise interaction term and a unary term, are coupled in the computation of the self-attention score, where the pairwise term contributes to learning the importance score for each feature interaction, while the unary term models the impact of one feature on all other features. We identify two factors, coupled gradient computation and shared transformations, impede the learning of both terms. To solve this problem, in this paper,we present a novel Disentangled Self-Attentive neural Network (DSAN) model for CTR prediction, which disentangles the two terms for facilitating learning feature interactions. We conduct extensive experiments framework using two real-world benchmark datasets. The results show that DSAN not only retains computational efficiency but obtains performance improvements over state-of-the-art baselines.



page 1

page 2

page 3

page 4


AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction

In this paper, we consider the Click-Through-Rate (CTR) prediction probl...

AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks

Click-through rate (CTR) prediction, which aims to predict the probabili...

Feature Interaction based Neural Network for Click-Through Rate Prediction

Click-Through Rate (CTR) prediction is one of the most important and cha...

FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine

Click through rate (CTR) estimation is a fundamental task in personalize...

Self-Attentive Associative Memory

Heretofore, neural networks with external memory are restricted to singl...

Learning High-Order Interactions via Targeted Pattern Search

Logistic Regression (LR) is a widely used statistical method in empirica...

Online Interaction Detection for Click-Through Rate Prediction

Click-Through Rate prediction aims to predict the ratio of clicks to imp...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.