Entertaining and Opinionated but Too Controlling: A Large-Scale User Study of an Open Domain Alexa Prize System

by   Kevin K. Bowden, et al.

Conversational systems typically focus on functional tasks such as scheduling appointments or creating todo lists. Instead we design and evaluate SlugBot (SB), one of 8 semifinalists in the 2018 AlexaPrize, whose goal is to support casual open-domain social inter-action. This novel application requires both broad topic coverage and engaging interactive skills. We developed a new technical approach to meet this demanding situation by crowd-sourcing novel content and introducing playful conversational strategies based on storytelling and games. We collected over 10,000 conversations during August 2018 as part of the Alexa Prize competition. We also conducted an in-lab follow-up qualitative evaluation. Over-all users found SB moderately engaging; conversations averaged 3.6 minutes and involved 26 user turns. However, users reacted very differently to different conversation subtypes. Storytelling and games were evaluated positively; these were seen as entertaining with predictable interactive structure. They also led users to impute personality and intelligence to SB. In contrast, search and general Chit-Chat induced coverage problems; here users found it hard to infer what topics SB could understand, with these conversations seen as being too system-driven. Theoretical and design implications suggest a move away from conversational systems that simply provide factual information. Future systems should be designed to have their own opinions with personal stories to share, and SB provides an example of how we might achieve this.


page 1

page 2

page 3

page 4


Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize

There has been an increased focus on creating conversational open-domain...

Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations

We present Chirpy Cardinal, an open-domain dialogue agent, as a research...

Dialogue Design and Management for Multi-Session Casual Conversation with Older Adults

We address the problem of designing a conversational avatar capable of a...

Would you Like to Talk about Sports Now? Towards Contextual Topic Suggestion for Open-Domain Conversational Agents

To hold a true conversation, an intelligent agent should be able to occa...

Ruuh: A Deep Learning Based Conversational Social Agent

Dialogue systems and conversational agents are becoming increasingly pop...

Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize

Building open domain conversational systems that allow users to have eng...

Audrey: A Personalized Open-Domain Conversational Bot

Conversational Intelligence requires that a person engage on information...

Please sign up or login with your details

Forgot password? Click here to reset