Extensible Grounding of Speech for Robot Instruction

07/31/2018
by   Jonathan Connell, et al.
8

Spoken language is a convenient interface for commanding a mobile robot. Yet for this to work a number of base terms must be grounded in perceptual and motor skills. We detail the language processing used on our robot ELI and explain how this grounding is performed, how it interacts with user gestures, and how it handles phenomena such as anaphora. More importantly, however, there are certain concepts which the robot cannot be preprogrammed with, such as the names of various objects in a household or the nature of specific tasks it may be requested to perform. In these cases it is vital that there exist a method for extending the grounding, essentially "learning by being told". We describe how this was successfully implemented for learning new nouns and verbs in a tabletop setting. Creating this language learning kernel may be the last explicit programming the robot ever needs - the core mechanism could eventually be used for imparting a vast amount of knowledge, much as a child learns from its parents and teachers.

READ FULL TEXT

page 5

page 6

page 7

page 8

page 10

page 13

page 16

page 20

research
09/29/2017

Symbol, Conversational, and Societal Grounding with a Toy Robot

Essential to meaningful interaction is grounding at the symbolic, conver...
research
08/14/2021

Sharing Cognition: Human Gesture and Natural Language Grounding Based Planning and Navigation for Indoor Robots

Cooperation among humans makes it easy to execute tasks and navigate sea...
research
09/05/2022

Trust in Language Grounding: a new AI challenge for human-robot teams

The challenge of language grounding is to fully understand natural langu...
research
07/14/2023

SGGNet^2: Speech-Scene Graph Grounding Network for Speech-guided Navigation

The spoken language serves as an accessible and efficient interface, ena...
research
08/17/2021

A Game Interface to Study Semantic Grounding in Text-Based Models

Can language models learn grounded representations from text distributio...
research
09/08/2003

What Is Working Memory and Mental Imagery? A Robot that Learns to Perform Mental Computations

This paper goes back to Turing (1936) and treats his machine as a cognit...

Please sign up or login with your details

Forgot password? Click here to reset