Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

11/22/2021
by   Pradip Pramanick, et al.
8

The utility of collocating robots largely depends on the easy and intuitive interaction mechanism with the human. If a robot accepts task instruction in natural language, first, it has to understand the user's intention by decoding the instruction. However, while executing the task, the robot may face unforeseeable circumstances due to the variations in the observed scene and therefore requires further user intervention. In this article, we present a system called Talk-to-Resolve (TTR) that enables a robot to initiate a coherent dialogue exchange with the instructor by observing the scene visually to resolve the impasse. Through dialogue, it either finds a cue to move forward in the original plan, an acceptable alternative to the original plan, or affirmation to abort the task altogether. To realize the possible stalemate, we utilize the dense captions of the observed scene and the given instruction jointly to compute the robot's next action. We evaluate our system based on a data set of initial instruction and situational scene pairs. Our system can identify the stalemate and resolve them with appropriate dialogue exchange with 82 systems are more natural (4.02 on average on a scale of 1 to 5) as compared to a state-of-the-art (3.08 on average).

READ FULL TEXT
research
03/05/2020

Your instruction may be crisp, but not clear to me!

The number of robots deployed in our daily surroundings is ever-increasi...
research
05/28/2018

Interactive Text2Pickup Network for Natural Language based Human-Robot Collaboration

In this paper, we propose the Interactive Text2Pickup (IT2P) network for...
research
08/23/2020

DeComplex: Task planning from complex natural instructions by a collocating robot

As the number of robots in our daily surroundings like home, office, res...
research
08/23/2020

Enabling human-like task identification from natural conversation

A robot as a coworker or a cohabitant is becoming mainstream day-by-day ...
research
07/21/2018

Consequences and Factors of Stylistic Differences in Human-Robot Dialogue

This paper identifies stylistic differences in instruction-giving observ...
research
10/01/2021

TEACh: Task-driven Embodied Agents that Chat

Robots operating in human spaces must be able to engage in natural langu...
research
10/04/2018

Balancing Efficiency and Coverage in Human-Robot Dialogue Collection

We describe a multi-phased Wizard-of-Oz approach to collecting human-rob...

Please sign up or login with your details

Forgot password? Click here to reset