Learning Optimal Tree Models Under Beam Search

by   Jingwei Zhuo, et al.

Retrieving relevant targets from an extremely large target set under computational limits is a common challenge for information retrieval and recommendation systems. Tree models, which formulate targets as leaves of a tree with trainable node-wise scorers, have attracted a lot of interests in tackling this challenge due to their logarithmic computational complexity in both training and testing. Tree-based deep models (TDMs) and probabilistic label trees (PLTs) are two representative kinds of them. Though achieving many practical successes, existing tree models suffer from the training-testing discrepancy, where the retrieval performance deterioration caused by beam search in testing is not considered in training. This leads to an intrinsic gap between the most relevant targets and those retrieved by beam search with even the optimally trained node-wise scorers. We take a first step towards understanding and analyzing this problem theoretically, and develop the concept of Bayes optimality under beam search and calibration under beam search as general analyzing tools for this purpose. Moreover, to eliminate the discrepancy, we propose a novel algorithm for learning optimal tree models under beam search. Experiments on both synthetic and real data verify the rationality of our theoretical analysis and demonstrate the superiority of our algorithm compared to state-of-the-art methods.


page 1

page 2

page 3

page 4


What Do You Get When You Cross Beam Search with Nucleus Sampling?

We combine beam search with the probabilistic pruning technique of nucle...

Enabling arbitrary translation objectives with Adaptive Tree Search

We introduce an adaptive tree search algorithm, that can find high-scori...

Context-aware Tree-based Deep Model for Recommender Systems

How to predict precise user preference and how to make efficient retriev...

Learning Beam Search Policies via Imitation Learning

Beam search is widely used for approximate decoding in structured predic...

Incremental Beam Manipulation for Natural Language Generation

The performance of natural language generation systems has improved subs...

Open Loop Execution of Tree-Search Algorithms

In the context of tree-search stochastic planning algorithms where a gen...

ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine

In a sponsored search engine, generative retrieval models are recently p...