Creatures great and SMAL: Recovering the shape and motion of animals from video

11/14/2018
by   Benjamin Biggs, et al.
8

We present a system to recover the 3D shape and motion of a wide variety of quadrupeds from video. The system comprises a machine learning front-end which predicts candidate 2D joint positions, a discrete optimization which finds kinematically plausible joint correspondences, and an energy minimization stage which fits a detailed 3D model to the image. In order to overcome the limited availability of motion capture training data from animals, and the difficulty of generating realistic synthetic training images, the system is designed to work on silhouette data. The joint candidate predictor is trained on synthetically generated silhouette images, and at test time, deep learning methods or standard video segmentation tools are used to extract silhouettes from real data. The system is tested on animal videos from several species, and shows accurate reconstructions of 3D shape and pose.

READ FULL TEXT

page 2

page 7

page 14

page 15

research
06/18/2021

hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern Recognition

In this paper we present our preliminary work on model-based behavioral ...
research
08/28/2018

DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth

Articulated hand pose and shape estimation is an important problem for v...
research
02/09/2019

3D Hand Shape and Pose from Images in the Wild

We present in this work the first end-to-end deep learning based method ...
research
04/16/2020

RGBD-Dog: Predicting Canine Pose from RGBD Sensors

The automatic extraction of animal 3D pose from images without markers i...
research
09/11/2018

Neural Animation and Reenactment of Human Actor Videos

We propose a method for generating (near) video-realistic animations of ...
research
03/21/2020

Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

We present a novel method for monocular hand shape and pose estimation a...
research
12/09/2017

A Deep Recurrent Framework for Cleaning Motion Capture Data

We present a deep, bidirectional, recurrent framework for cleaning noisy...

Please sign up or login with your details

Forgot password? Click here to reset