Low-Latency Immersive 6D Televisualization with Spherical Rendering

09/23/2021
by   Max Schwarz, et al.
0

We present a method for real-time stereo scene capture and remote VR visualization that allows a human operator to freely move their head and thus intuitively control their perspective during teleoperation. The stereo camera is mounted on a 6D robotic arm, which follows the operator's head pose. Existing VR teleoperation systems either induce high latencies on head movements, leading to motion sickness, or use scene reconstruction methods to allow re-rendering of the scene from different perspectives, which cannot handle dynamic scenes effectively. Instead, we present a decoupled approach which renders captured camera images as spheres, assuming constant distance. This allows very fast re-rendering on head pose changes while keeping the resulting temporary distortions during head translations small. We present qualitative examples, quantitative results in the form of lab experiments and a small user study, showing that our method outperforms other visualization methods.

READ FULL TEXT

page 1

page 2

page 3

page 5

research
09/28/2021

NimbRo Avatar: Interactive Immersive Telepresence with Force-Feedback Telemanipulation

Robotic avatars promise immersive teleoperation with human-like manipula...
research
08/23/2023

NimbRo wins ANA Avatar XPRIZE Immersive Telepresence Competition: Human-Centric Evaluation and Lessons Learned

Robotic avatar systems can enable immersive telepresence with locomotion...
research
08/14/2020

MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images

We introduce a method to convert stereo 360 (omnidirectional stereo) ima...
research
10/27/2021

Smooth head tracking for virtual reality applications

In this work, we propose a new head-tracking solution for human-machine ...
research
04/24/2023

VR Facial Animation for Immersive Telepresence Avatars

VR Facial Animation is necessary in applications requiring clear view of...
research
08/08/2019

A VR System for Immersive Teleoperation and Live Exploration with a Mobile Robot

Applications like disaster management and industrial inspection often re...
research
09/14/2018

MoSculp: Interactive Visualization of Shape and Time

We present a system that allows users to visualize complex human motion ...

Please sign up or login with your details

Forgot password? Click here to reset