Solving Visual Object Ambiguities when Pointing: An Unsupervised Learning Approach

12/13/2019
by   Doreen Jirak, et al.
16

Whenever we are addressing a specific object or refer to a certain spatial location, we are using referential or deictic gestures usually accompanied by some verbal description. Especially pointing gestures are necessary to dissolve ambiguities in a scene and they are of crucial importance when verbal communication may fail due to environmental conditions or when two persons simply do not speak the same language. With the currently increasing advances of humanoid robots and their future integration in domestic domains, the development of gesture interfaces complementing human-robot interaction scenarios is of substantial interest. The implementation of an intuitive gesture scenario is still challenging because both the pointing intention and the corresponding object have to be correctly recognized in real-time. The demand increases when considering pointing gestures in a cluttered environment, as is the case in households. Also, humans perform pointing in many different ways and those variations have to be captured. Research in this field often proposes a set of geometrical computations which do not scale well with the number of gestures and objects, use specific markers or a predefined set of pointing directions. In this paper, we propose an unsupervised learning approach to model the distribution of pointing gestures using a growing-when-required (GWR) network. We introduce an interaction scenario with a humanoid robot and define so-called ambiguity classes. Our implementation for the hand and object detection is independent of any markers or skeleton models, thus it can be easily reproduced. Our evaluation comparing a baseline computer vision approach with our GWR model shows that the pointing-object association is well learned even in cases of ambiguities resulting from close object proximity.

READ FULL TEXT

page 7

page 9

page 10

page 13

page 14

page 19

page 22

page 32

research
09/21/2021

A Proposed Set of Communicative Gestures for Human Robot Interaction and an RGB Image-based Gesture Recognizer Implemented in ROS

We propose a set of communicative gestures and develop a gesture recogni...
research
06/22/2022

Real-Time Online Skeleton Extraction and Gesture Recognition on Pepper

We present a multi-stage pipeline for simple gesture recognition. The no...
research
03/08/2023

Communicating human intent to a robotic companion by multi-type gesture sentences

Human-Robot collaboration in home and industrial workspaces is on the ri...
research
09/20/2022

HyperPalm: DNN-based hand gesture recognition interface for intelligent communication with quadruped robot in 3D space

Nowadays, autonomous mobile robots support people in many areas where hu...
research
09/06/2023

Gesture-Informed Robot Assistance via Foundation Models

Gestures serve as a fundamental and significant mode of non-verbal commu...
research
07/25/2022

Continuous ErrP detections during multimodal human-robot interaction

Human-in-the-loop approaches are of great importance for robot applicati...
research
03/04/2020

Learning,Generating and Adapting Wave Gestures for Expressive Human-Robot Interaction

This study proposes a novel imitation learning approach for the stochast...

Please sign up or login with your details

Forgot password? Click here to reset