Learning to Infer User Hidden States for Online Sequential Advertising

by   Zhaoqing Peng, et al.

To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important. The lack of interpretability in existing deep reinforcement learning methods makes it not easy to understand, diagnose and further optimize the strategy. In this paper, we propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer's purchase intent which is, however, unobservable (called hidden states). In this paper, we model this intention as a latent variable and formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the underlying intents are inferred based on the observable behaviors. Large-scale industrial offline and online experiments demonstrate our method's superior performance over several baselines. The inferred hidden states are analyzed, and the results prove the rationality of our inference.


page 1

page 2

page 3

page 4


Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds

Most e-commerce product feeds provide blended results of advertised prod...

Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising

In E-commerce, advertising is essential for merchants to reach their tar...

Multi-objective Optimization of Notifications Using Offline Reinforcement Learning

Mobile notification systems play a major role in a variety of applicatio...

Blockwise Sequential Model Learning for Partially Observable Reinforcement Learning

This paper proposes a new sequential model learning architecture to solv...

Technical Report: Distribution Temporal Logic: Combining Correctness with Quality of Estimation

We present a new temporal logic called Distribution Temporal Logic (DTL)...

A Deep Prediction Network for Understanding Advertiser Intent and Satisfaction

For e-commerce platforms such as Taobao and Amazon, advertisers play an ...

Incrementality Bidding via Reinforcement Learning under Mixed and Delayed Rewards

Incrementality, which is used to measure the causal effect of showing an...