Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction

01/11/2021
by   Yanqiao Zhu, et al.
0

Click-through rate (CTR) prediction, which aims to predict the probability that whether of a user will click on an item, is an essential task for many online applications. Due to the nature of data sparsity and high dimensionality in CTR prediction, a key to making effective prediction is to model high-order feature interactions among feature fields. To explicitly model high-order feature interactions, an efficient way is to stack multihead self-attentive neural networks, which has achieved promising performance. However, one problem of the vanilla self-attentive network is that two terms, a whitened pairwise interaction term and a unary term, are coupled in the computation of the self-attention score, where the pairwise term contributes to learning the importance score for each feature interaction, while the unary term models the impact of one feature on all other features. We identify two factors, coupled gradient computation and shared transformations, impede the learning of both terms. To solve this problem, in this paper,we present a novel Disentangled Self-Attentive neural Network (DSAN) model for CTR prediction, which disentangles the two terms for facilitating learning feature interactions. We conduct extensive experiments framework using two real-world benchmark datasets. The results show that DSAN not only retains computational efficiency but obtains performance improvements over state-of-the-art baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/20/2020

AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction

In this paper, we consider the Click-Through-Rate (CTR) prediction probl...
research
10/29/2018

AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks

Click-through rate (CTR) prediction, which aims to predict the probabili...
research
02/16/2020

Generalized Embedding Machines for Recommender Systems

Factorization machine (FM) is an effective model for feature-based recom...
research
05/15/2019

FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine

Click through rate (CTR) estimation is a fundamental task in personalize...
research
04/21/2023

EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction

Learning effective high-order feature interactions is very crucial in th...
research
02/10/2020

Self-Attentive Associative Memory

Heretofore, neural networks with external memory are restricted to singl...
research
02/23/2021

Learning High-Order Interactions via Targeted Pattern Search

Logistic Regression (LR) is a widely used statistical method in empirica...

Please sign up or login with your details

Forgot password? Click here to reset