Kalman Filter-based Head Motion Prediction for Cloud-based Mixed Reality

07/28/2020
by   Serhan Gül, et al.
0

Volumetric video allows viewers to experience highly-realistic 3D content with six degrees of freedom in mixed reality (MR) environments. Rendering complex volumetric videos can require a prohibitively high amount of computational power for mobile devices. A promising technique to reduce the computational burden on mobile devices is to perform the rendering at a cloud server. However, cloud-based rendering systems suffer from an increased interaction (motion-to-photon) latency that may cause registration errors in MR environments. One way of reducing the effective latency is to predict the viewer's head pose and render the corresponding view from the volumetric video in advance. In this paper, we design a Kalman filter for head motion prediction in our cloud-based volumetric video streaming system. We analyze the performance of our approach using recorded head motion traces and compare its performance to an autoregression model for different prediction intervals (look-ahead times). Our results show that the Kalman filter can predict head orientations 0.5 degrees more accurately than the autoregression model for a look-ahead time of 60 ms.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/17/2020

Low-latency Cloud-based Volumetric Video Streaming Using Head Motion Prediction

Volumetric video is an emerging key technology for immersive representat...
research
03/05/2020

Cloud Rendering-based Volumetric Video Streaming System for Mixed Reality Services

Volumetric video is an emerging technology for immersive representation ...
research
01/17/2020

Low Latency Volumetric Video Edge Cloud Streaming

Volumetric video is an emerging key technology for immersive representat...
research
03/22/2017

Adaptive User Perspective Rendering for Handheld Augmented Reality

Handheld Augmented Reality commonly implements some variant of magic len...
research
01/26/2022

Mixed reality hologram slicer (mxdR-HS): a marker-less tangible user interface for interactive holographic volume visualization

Mixed reality head-mounted displays (mxdR-HMD) have the potential to vis...
research
03/15/2023

Progressive Frame Patching for FoV-based Point Cloud Video Streaming

Immersive multimedia applications, such as Virtual, Augmented and Mixed ...
research
07/11/2021

Interpretable Deep Feature Propagation for Early Action Recognition

Early action recognition (action prediction) from limited preliminary ob...

Please sign up or login with your details

Forgot password? Click here to reset