DeepAI AI Chat
Log In Sign Up

Improving Grounded Natural Language Understanding through Human-Robot Dialog

by   Jesse Thomason, et al.

Natural language understanding for robotics can require substantial domain- and platform-specific engineering. For example, for mobile robots to pick-and-place objects in an environment to satisfy human commands, we can specify the language humans use to issue such commands, and connect concept words like red can to physical object properties. One way to alleviate this engineering for a new domain is to enable robots in human environments to adapt dynamically---continually learning new language constructions and perceptual concepts. In this work, we present an end-to-end pipeline for translating natural language commands to discrete robot actions, and use clarification dialogs to jointly improve language parsing and concept grounding. We train and evaluate this agent in a virtual setting on Amazon Mechanical Turk, and we transfer the learned agent to a physical robot platform to demonstrate it in the real world.


page 1

page 2

page 4

page 6


Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions

The speed and accuracy with which robots are able to interpret natural l...

Interactive Natural Language-based Person Search

In this work, we consider the problem of searching people in an unconstr...

Language Grounding with 3D Objects

Seemingly simple natural language requests to a robot are generally unde...

Towards VEsNA, a Framework for Managing Virtual Environments via Natural Language Agents

Automating a factory where robots are involved is neither trivial nor ch...

Processing Natural Language About Ongoing Actions

Actions may not proceed as planned; they may be interrupted, resumed or ...

Improving Natural Language Interaction with Robots Using Advice

Over the last few years, there has been growing interest in learning mod...

Interactive Learning of State Representation through Natural Language Instruction and Explanation

One significant simplification in most previous work on robot learning i...