CobotAR: Interaction with Robots using Omnidirectionally Projected Image and DNN-based Gesture Recognition

10/20/2021
by   Nazarova Elena, et al.
0

Several technological solutions supported the creation of interfaces for Augmented Reality (AR) multi-user collaboration in the last years. However, these technologies require the use of wearable devices. We present CobotAR - a new AR technology to achieve the Human-Robot Interaction (HRI) by gesture recognition based on Deep Neural Network (DNN) - without an extra wearable device for the user. The system allows users to have a more intuitive experience with robotic applications using just their hands. The CobotAR system assumes the AR spatial display created by a mobile projector mounted on a 6 DoF robot. The proposed technology suggests a novel way of interaction with machines to achieve safe, intuitive, and immersive control mediated by a robotic projection system and DNN-based algorithm. We conducted the experiment with several parameters assessment during this research, which allows the users to define the positives and negatives of the new approach. The mental demand of CobotAR system is twice less than Wireless Gamepad and by 16% less than Teach Pendant.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 6

research
11/02/2022

CobotTouch: AR-based Interface with Fingertip-worn Tactile Display for Immersive Operation/Control of Collaborative Robots

Complex robotic tasks require human collaboration to benefit from their ...
research
11/07/2020

ZoomTouch: Multi-User Remote Robot Control in Zoom by DNN-based Gesture Recognition

We present ZoomTouch - a breakthrough technology for multi-user control ...
research
09/06/2019

AR-based interaction for safe human-robot collaborative manufacturing

Industrial standards define safety requirements for Human-Robot Collabor...
research
10/29/2021

ARviz – An Augmented Reality-enabled Visualization Platform for ROS Applications

Current robot interfaces such as teach pendants and 2D screen displays u...
research
12/15/2017

Holoscopic 3D Micro-Gesture Database for Wearable Device Interaction

With the rapid development of augmented reality (AR) and virtual reality...
research
06/29/2023

AI-Powered Interfaces for Extended Reality to support Remote Maintenance

High-end components that conduct complicated tasks automatically are a p...
research
04/04/2019

Referencing between a Head-Mounted Device and Robotic Manipulators

Having a precise and robust transformation between the robot coordinate ...

Please sign up or login with your details

Forgot password? Click here to reset