DeepAI AI Chat
Log In Sign Up

Correcting for Selection Bias in Learning-to-rank Systems

by   Zohreh Ovaisi, et al.
Cal Poly
University of Illinois at Chicago

Click data collected by modern recommendation systems are an important source of observational data that can be utilized to train learning-to-rank (LTR) systems. However, these data suffer from a number of biases that can result in poor performance for LTR systems. Recent methods for bias correction in such systems mostly focus on position bias, the fact that higher ranked results (e.g., top search engine results) are more likely to be clicked even if they are not the most relevant results given a user's query. Less attention has been paid to correcting for selection bias, which occurs because clicked documents are reflective of what documents have been shown to the user in the first place. Here, we propose new counterfactual approaches which adapt Heckman's two-stage method and accounts for selection and position bias in LTR systems. Our empirical evaluation shows that our proposed methods are much more robust to noise and have better accuracy compared to existing unbiased LTR algorithms, especially when there is moderate to no position bias.


page 1

page 2

page 3

page 4


Handling Position Bias for Unbiased Learning to Rank in Hotels Search

Nowadays, search ranking and recommendation systems rely on a lot of dat...

Mixture-Based Correction for Position and Trust Bias in Counterfactual Learning to Rank

In counterfactual learning to rank (CLTR) user interactions are used as ...

Reinforcement Online Learning to Rank with Unbiased Reward Shaping

Online learning to rank (OLTR) aims to learn a ranker directly from impl...

Unbiased Learning to Rank via Propensity Ratio Scoring

Implicit feedback, such as user clicks, is a major source of supervision...

Unbiased Learning to Rank: Counterfactual and Online Approaches

This tutorial covers and contrasts the two main methodologies in unbiase...

Counterfactual Learning to Rank using Heterogeneous Treatment Effect Estimation

Learning-to-Rank (LTR) models trained from implicit feedback (e.g. click...

Incorporating Vision Bias into Click Models for Image-oriented Search Engine

Most typical click models assume that the probability of a document to b...