Unbiased Learning to Rank: Counterfactual and Online Approaches

07/16/2019
by   Harrie Oosterhuis, et al.
0

This tutorial covers and contrasts the two main methodologies in unbiased Learning to Rank (LTR): Counterfactual LTR and Online LTR. There has long been an interest in LTR from user interactions, however, this form of implicit feedback is very biased. In recent years, unbiased LTR methods have been introduced to remove the effect of different types of bias caused by user-behavior in search. For instance, a well addressed type of bias is position bias: the rank at which a document is displayed heavily affects the interactions it receives. Counterfactual LTR methods deal with such types of bias by learning from historical interactions while correcting for the effect of the explicitly modelled biases. Online LTR does not use an explicit user model, in contrast, it learns through an interactive process where randomized results are displayed to the user. Through randomization the effect of different types of bias can be removed from the learning process. Though both methodologies lead to unbiased LTR, their approaches differ considerably, furthermore, so do their theoretical guarantees, empirical results, effects on the user experience during learning, and applicability. Consequently, for practitioners the choice between the two is very substantial. By providing an overview of both approaches and contrasting them, we aim to provide an essential guide to unbiased LTR so as to aid in understanding and choosing between methodologies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/15/2019

To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions

Learning to Rank (LTR) from user interactions is challenging as user fee...
research
04/01/2022

Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach

In this paper we study how to effectively exploit implicit feedback in D...
research
06/24/2022

Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank

Click-based learning to rank (LTR) tackles the mismatch between click fr...
research
05/21/2020

Accelerated Convergence for Counterfactual Learning to Rank

Counterfactual Learning to Rank (LTR) algorithms learn a ranking model f...
research
07/08/2020

Unbiased Lift-based Bidding System

Conventional bidding strategies for online display ad auction heavily re...
research
10/19/2022

Whole Page Unbiased Learning to Rank

The page presentation biases in the information retrieval system, especi...
research
05/25/2020

Cascade Model-based Propensity Estimation for Counterfactual Learning to Rank

Unbiased CLTR requires click propensities to compensate for the differen...

Please sign up or login with your details

Forgot password? Click here to reset