A Conceptual Framework for Implicit Evaluation of Conversational Search Interfaces

by   Abhishek Kaushik, et al.

Conversational search (CS) has recently become a significant focus of the information retrieval (IR) research community. Multiple studies have been conducted which explore the concept of conversational search. Understanding and advancing research in CS requires careful and detailed evaluation. Existing CS studies have been limited to evaluation based on simple user feedback on task completion. We propose a CS evaluation framework which includes multiple dimensions: search experience, knowledge gain, software usability, cognitive load and user experience, based on studies of conversational systems and IR. We introduce these evaluation criteria and propose their use in a framework for the evaluation of CS systems.


page 1

page 2

page 3

page 4


Exploiting Simulated User Feedback for Conversational Search: Ranking, Rewriting, and Beyond

This research aims to explore various methods for assessing user feedbac...

Query Performance Prediction: From Ad-hoc to Conversational Search

Query performance prediction (QPP) is a core task in information retriev...

Comparing Conventional and Conversational Search Interaction using Implicit Evaluation Methods

Conversational search applications offer the prospect of improved user e...

Cognitive Simplification Operations Improve Text Simplification

Text Simplification (TS) is the task of converting a text into a form th...

Design and Performance Analysis of Hardware Realization of 3GPP Physical Layer for 5G Cell Search

5G Cell Search (CS) is the first step for user equipment (UE) to initiat...

Taking Search to Task

The importance of tasks in information retrieval (IR) has been long argu...

Role of Attentive History Selection in Conversational Information Seeking

The rise of intelligent assistant systems like Siri and Alexa have led t...

Please sign up or login with your details

Forgot password? Click here to reset