Analysing Mixed Initiatives and Search Strategies during Conversational Search

09/13/2021
by   Mohammad Aliannejadi, et al.
0

Information seeking conversations between users and Conversational Search Agents (CSAs) consist of multiple turns of interaction. While users initiate a search session, ideally a CSA should sometimes take the lead in the conversation by obtaining feedback from the user by offering query suggestions or asking for query clarifications i.e. mixed initiative. This creates the potential for more engaging conversational searches, but substantially increases the complexity of modelling and evaluating such scenarios due to the large interaction space coupled with the trade-offs between the costs and benefits of the different interactions. In this paper, we present a model for conversational search – from which we instantiate different observed conversational search strategies, where the agent elicits: (i) Feedback-First, or (ii) Feedback-After. Using 49 TREC WebTrack Topics, we performed an analysis comparing how well these different strategies combine with different mixed initiative approaches: (i) Query Suggestions vs. (ii) Query Clarifications. Our analysis reveals that there is no superior or dominant combination, instead it shows that query clarifications are better when asked first, while query suggestions are better when asked after presenting results. We also show that the best strategy and approach depends on the trade-offs between the relative costs between querying and giving feedback, the performance of the initial query, the number of assessments per query, and the total amount of gain required. While this work highlights the complexities and challenges involved in analyzing CSAs, it provides the foundations for evaluating conversational strategies and conversational search agents in batch/offline settings.

READ FULL TEXT

page 3

page 4

research
01/21/2022

Towards Building Economic Models of Conversational Search

Various conceptual and descriptive models of conversational search have ...
research
02/18/2022

Personalization Trade-offs in Designing a Dialogue-based Information System for Support-Seeking of Sexual Violence Survivors

The lack of reliable, personalized information often complicates sexual ...
research
12/27/2018

QRFA: A Data-Driven Model of Information-Seeking Dialogues

Understanding the structure of interaction processes helps us to improve...
research
09/17/2017

Reinforcement Learning Based Conversational Search Assistant

In this work, we develop an end-to-end Reinforcement Learning based arch...
research
05/25/2020

An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues

The ability to engage in mixed-initiative interaction is one of the core...
research
08/03/2023

Investigating Reinforcement Learning for Communication Strategies in a Task-Initiative Setting

Many conversational domains require the system to present nuanced inform...
research
08/05/2022

Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls

The promise of interaction between intelligent conversational agents and...

Please sign up or login with your details

Forgot password? Click here to reset