User Model-Based Intent-Aware Metrics for Multilingual Search Evaluation

12/13/2016
by   Alexey Drutsa, et al.
0

Despite the growing importance of multilingual aspect of web search, no appropriate offline metrics to evaluate its quality are proposed so far. At the same time, personal language preferences can be regarded as intents of a query. This approach translates the multilingual search problem into a particular task of search diversification. Furthermore, the standard intent-aware approach could be adopted to build a diversified metric for multilingual search on the basis of a classical IR metric such as ERR. The intent-aware approach estimates user satisfaction under a user behavior model. We show however that the underlying user behavior models is not realistic in the multilingual case, and the produced intent-aware metric do not appropriately estimate the user satisfaction. We develop a novel approach to build intent-aware user behavior models, which overcome these limitations and convert to quality metrics that better correlate with standard online metrics of user satisfaction.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/25/2023

An Intent Taxonomy of Legal Case Retrieval

Legal case retrieval is a special Information Retrieval (IR) task focusi...
research
11/27/2017

Why People Search for Images using Web Search Engines

What are the intents or goals behind human interactions with image searc...
research
07/04/2022

Still Haven't Found What You're Looking For – Detecting the Intent of Web Search Missions from User Interaction Features

Web search is among the most frequent online activities. Whereas traditi...
research
06/20/2018

Multi-Layer Ensembling Techniques for Multilingual Intent Classification

In this paper we determine how multi-layer ensembling improves performan...
research
06/06/2023

Generate-then-Retrieve: Intent-Aware FAQ Retrieval in Product Search

Customers interacting with product search engines are increasingly formu...
research
09/15/2023

Multilingual Sentence-Level Semantic Search using Meta-Distillation Learning

Multilingual semantic search is the task of retrieving relevant contents...
research
06/14/2023

Contextual Font Recommendations based on User Intent

Adobe Fonts has a rich library of over 20,000 unique fonts that Adobe us...

Please sign up or login with your details

Forgot password? Click here to reset