Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos

11/29/2021
by   Denys Rozumnyi, et al.
0

We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. To this end, we model the blurred appearance of a fast moving object in a generative fashion by parametrizing its 3D position, rotation, velocity, acceleration, bounces, shape, and texture over the duration of a predefined time window spanning multiple frames. Using differentiable rendering, we are able to estimate all parameters by minimizing the pixel-wise reprojection error to the input video via backpropagating through a rendering pipeline that accounts for motion blur by averaging the graphics output over short time intervals. For that purpose, we also estimate the camera exposure gap time within the same optimization. To account for abrupt motion changes like bounces, we model the motion trajectory as a piece-wise polynomial, and we are able to estimate the specific time of the bounce at sub-frame accuracy. Experiments on established benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.

READ FULL TEXT

page 1

page 4

page 5

page 7

page 8

research
11/25/2019

Sub-frame Appearance and 6D Pose Estimation of Fast Moving Objects

We propose a novel method that tracks fast moving objects, mainly non-un...
research
06/16/2021

Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects

We address the novel task of jointly reconstructing the 3D shape, textur...
research
04/28/2023

3D shape reconstruction of semi-transparent worms

3D shape reconstruction typically requires identifying object features o...
research
12/05/2022

Minimum Latency Deep Online Video Stabilization

We present a novel camera path optimization framework for the task of on...
research
03/30/2023

Human from Blur: Human Pose Tracking from Blurry Images

We propose a method to estimate 3D human poses from substantially blurre...
research
08/14/2018

Moving Object Segmentation in Jittery Videos by Stabilizing Trajectories Modeled in Kendall's Shape Space

Moving Object Segmentation is a challenging task for jittery/wobbly vide...
research
05/09/2019

Intra-frame Object Tracking by Deblatting

Objects moving at high speed along complex trajectories often appear in ...

Please sign up or login with your details

Forgot password? Click here to reset