DeepAI AI Chat
Log In Sign Up

A VR System for Immersive Teleoperation and Live Exploration with a Mobile Robot

by   Patrick Stotko, et al.

Applications like disaster management and industrial inspection often require experts to enter contaminated places. To circumvent the need for physical presence, it is desirable to generate a fully immersive individual live teleoperation experience. However, standard video-based approaches suffer from a limited degree of immersion and situation awareness due to the restriction to the camera view, which impacts the navigation. In this paper, we present a novel VR-based practical system for immersive robot teleoperation and scene exploration. While being operated through the scene, a robot captures RGB-D data that is streamed to a SLAM-based live multi-client telepresence system. Here, a global 3D model of the already captured scene parts is reconstructed and streamed to the individual remote user clients where the rendering for e.g. head-mounted display devices (HMDs) is performed. We introduce a novel lightweight robot client component which transmits robot-specific data and enables a quick integration into existing robotic systems. This way, in contrast to first-person exploration systems, the operators can explore and navigate in the remote site completely independent of the current position and view of the capturing robot, complementing traditional input devices for teleoperation. We provide a proof-of-concept implementation and demonstrate the capabilities as well as the performance of our system regarding interactive object measurements and bandwidth-efficient data streaming and visualization. Furthermore, we show its benefits over purely video-based teleoperation in a user study revealing a higher degree of situation awareness and a more precise navigation in challenging environments.


page 1

page 5

page 6

page 7


Unwinding Rotations Improves User Comfort with Immersive Telepresence Robots

We propose unwinding the rotations experienced by the user of an immersi...

SLAMCast: Large-Scale, Real-Time 3D Reconstruction and Streaming for Immersive Multi-Client Live Telepresence

Real-time 3D scene reconstruction from RGB-D sensor data, as well as the...

Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence

Sharing live telepresence experiences for teleconferencing or remote col...

Online Next-Best-View Planner for 3D-Exploration and Inspection With a Mobile Manipulator Robot

Robotic systems performing end-user oriented autonomous exploration can ...

A Flexible Framework for Virtual Omnidirectional Vision to Improve Operator Situation Awareness

During teleoperation of a mobile robot, providing good operator situatio...

Low-Latency Immersive 6D Televisualization with Spherical Rendering

We present a method for real-time stereo scene capture and remote VR vis...