Two-stage Voice Application Recommender System for Unhandled Utterances in Intelligent Personal Assistant

10/19/2021
by   Wei Xiao, et al.
0

Intelligent personal assistants (IPA) enable voice applications that facilitate people's daily tasks. However, due to the complexity and ambiguity of voice requests, some requests may not be handled properly by the standard natural language understanding (NLU) component. In such cases, a simple reply like "Sorry, I don't know" hurts the user's experience and limits the functionality of IPA. In this paper, we propose a two-stage shortlister-reranker recommender system to match third-party voice applications (skills) to unhandled utterances. In this approach, a skill shortlister is proposed to retrieve candidate skills from the skill catalog by calculating both lexical and semantic similarity between skills and user requests. We also illustrate how to build a new system by using observed data collected from a baseline rule-based system, and how the exposure biases can generate discrepancy between offline and human metrics. Lastly, we present two relabeling methods that can handle the incomplete ground truth, and mitigate exposure bias. We demonstrate the effectiveness of our proposed system through extensive offline experiments. Furthermore, we present online A/B testing results that show a significant boost on user experience satisfaction.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/03/2019

Evaluating Voice Skills by Design Guidelines Using an Automatic Voice Crawler

Currently, adaptive voice applications supported by voice assistants (VA...
research
12/16/2022

SkillFence: A Systems Approach to Practically Mitigating Voice-Based Confusion Attacks

Voice assistants are deployed widely and provide useful functionality. H...
research
06/22/2022

When It's Not Worth the Paper It's Written On: A Provocation on the Certification of Skills in the Alexa and Google Assistant Ecosystems

The increasing reach and functionality of voice assistants has allowed t...
research
04/30/2020

Learning to Rank Intents in Voice Assistants

Voice Assistants aim to fulfill user requests by choosing the best inten...
research
02/09/2020

Meet Malexa, Alexa's Malicious Twin: Malware-Induced Misperception Through Intelligent Voice Assistants

This paper reports the findings of a study where users (N=220) interacte...
research
12/06/2019

What Do You Mean I'm Funny? Personalizing the Joke Skill of a Voice-Controlled Virtual Assistant

A considerable part of the success experienced by Voice-controlled virtu...
research
05/12/2021

"Alexa, what do you do for fun?" Characterizing playful requests with virtual assistants

Virtual assistants such as Amazon's Alexa, Apple's Siri, Google Home, an...

Please sign up or login with your details

Forgot password? Click here to reset