Enhanced Visual Feedback with Decoupled Viewpoint Control in Immersive Humanoid Robot Teleoperation using SLAM

by   Yang Chen, et al.

In immersive humanoid robot teleoperation, there are three main shortcomings that can alter the transparency of the visual feedback: the lag between the motion of the operator's and robot's head due to network communication delays or slow robot joint motion. This latency could cause a noticeable delay in the visual feedback, which jeopardizes the embodiment quality, can cause dizziness, and affects the interactivity resulting in operator frequent motion pauses for the visual feedback to settle; (ii) the mismatch between the camera's and the headset's field-of-views (FOV), the former having generally a lower FOV; and (iii) a mismatch between human's and robot's range of motions of the neck, the latter being also generally lower. In order to leverage these drawbacks, we developed a decoupled viewpoint control solution for a humanoid platform which allows visual feedback with low-latency and artificially increases the camera's FOV range to match that of the operator's headset. Our novel solution uses SLAM technology to enhance the visual feedback from a reconstructed mesh, complementing the areas that are not covered by the visual feedback from the robot. The visual feedback is presented as a point cloud in real-time to the operator. As a result, the operator is fed with real-time vision from the robot's head orientation by observing the pose of the point cloud. Balancing this kind of awareness and immersion is important in virtual reality based teleoperation, considering the safety and robustness of the control system. An experiment shows the effectiveness of our solution.


page 1

page 5

page 7


Prescient teleoperation of humanoid robots

Humanoid robots could be versatile and intuitive human avatars that oper...

iCub3 Avatar System

We present an avatar system that enables a human operator to visit a rem...

The Effects of Visual and Control Latency on Piloting a Quadcopter using a Head-Mounted Display

Recent research has proposed teleoperation of robotic and aerial vehicle...

Facilitating Self-monitored Physical Rehabilitation with Virtual Reality and Haptic feedback

Physical rehabilitation is essential to recovery from joint replacement ...

Feedback and Control of Dynamics and Robotics using Augmented Reality

Human-machine interaction (HMI) and human-robot interaction (HRI) can as...

Design of a user-friendly control system for planetary rovers with CPS feature

In this paper, we present a user-friendly planetary rover's control syst...

Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach

Haptic feedback can improve safety of teleoperated robots when situation...

Please sign up or login with your details

Forgot password? Click here to reset