UserBERT: Modeling Long- and Short-Term User Preferences via Self-Supervision

02/14/2022
by   Tianyu Li, et al.
0

E-commerce platforms generate vast amounts of customer behavior data, such as clicks and purchases, from millions of unique users every day. However, effectively using this data for behavior understanding tasks is challenging because there are usually not enough labels to learn from all users in a supervised manner. This paper extends the BERT model to e-commerce user data for pre-training representations in a self-supervised manner. By viewing user actions in sequences as analogous to words in sentences, we extend the existing BERT model to user behavior data. Further, our model adopts a unified structure to simultaneously learn from long-term and short-term user behavior, as well as user attributes. We propose methods for the tokenization of different types of user behavior sequences, the generation of input representation vectors, and a novel pretext task to enable the pre-trained model to learn from its own input, eliminating the need for labeled training data. Extensive experiments demonstrate that the learned representations result in significant improvements when transferred to three different real-world tasks, particularly compared to task-specific modeling and multi-task representation learning

READ FULL TEXT
research
10/04/2020

PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervision

User modeling is critical for many personalized web services. Many exist...
research
11/26/2018

Attentive Long Short-Term Preference Modeling for Personalized Product Search

E-commerce users may expect different products even for the same query, ...
research
06/13/2021

InfoBehavior: Self-supervised Representation Learning for Ultra-long Behavior Sequence via Hierarchical Grouping

E-commerce companies have to face abnormal sellers who sell potentially-...
research
06/15/2022

Robust Attack Graph Generation

We present a method to learn automaton models that are more robust to in...
research
09/18/2021

Interest-oriented Universal User Representation via Contrastive Learning

User representation is essential for providing high-quality commercial s...
research
05/28/2018

Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks

Tasks such as search and recommendation have become increas- ingly impor...
research
10/20/2021

Learning Universal User Representations via Self-Supervised Lifelong Behaviors Modeling

Universal user representation is an important research topic in industry...

Please sign up or login with your details

Forgot password? Click here to reset