IntenT5: Search Result Diversification using Causal Language Models

08/09/2021
by   Sean MacAvaney, et al.
0

Search result diversification is a beneficial approach to overcome under-specified queries, such as those that are ambiguous or multi-faceted. Existing approaches often rely on massive query logs and interaction data to generate a variety of possible query intents, which then can be used to re-rank documents. However, relying on user interaction data is problematic because one first needs a massive user base to build a sufficient log; public query logs are insufficient on their own. Given the recent success of causal language models (such as the Text-To-Text Transformer (T5) model) at text generation tasks, we explore the capacity of these models to generate potential query intents. We find that to encourage diversity in the generated queries, it is beneficial to adapt the model by including a new Distributional Causal Language Modeling (DCLM) objective during fine-tuning and a representation replacement during inference. Across six standard evaluation benchmarks, we find that our method (which we call IntenT5) improves search result diversity and attains (and sometimes exceeds) the diversity obtained when using query suggestions based on a proprietary query log. Our analysis shows that our approach is most effective for multi-faceted queries and is able to generalize effectively to queries that were unseen in training data.

READ FULL TEXT
research
08/31/2023

Context Aware Query Rewriting for Text Rankers using LLM

Query rewriting refers to an established family of approaches that are a...
research
12/28/2021

Query Suggestion for Click-Absent Queries in Enterprise Search

Creating alternative queries, also known as query suggestion, has been p...
research
04/02/2023

The Archive Query Log: Mining Millions of Search Result Pages of Hundreds of Search Engines from 25 Years of Web Archives

The Archive Query Log (AQL) is a previously unused, comprehensive query ...
research
08/06/2020

Efficient Neural Query Auto Completion

Query Auto Completion (QAC), as the starting point of information retrie...
research
04/14/2020

Query-Variant Advertisement Text Generation with Association Knowledge

Advertising is an important revenue source for many companies. However, ...
research
10/28/2022

You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LM

Retrieval-enhanced language models (LMs), which condition their predicti...
research
07/13/2023

Prompts Should not be Seen as Secrets: Systematically Measuring Prompt Extraction Attack Success

The generations of large language models are commonly controlled through...

Please sign up or login with your details

Forgot password? Click here to reset