3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning

07/27/2020
by   Xiangyu Xu, et al.
7

3D human shape and pose estimation from monocular images has been an active area of research in computer vision, having a substantial impact on the development of new applications, from activity recognition to creating virtual avatars. Existing deep learning methods for 3D human shape and pose estimation rely on relatively high-resolution input images; however, high-resolution visual content is not always available in several practical scenarios such as video surveillance and sports broadcasting. Low-resolution images in real scenarios can vary in a wide range of sizes, and a model trained in one resolution does not typically degrade gracefully across resolutions. Two common approaches to solve the problem of low-resolution input are applying super-resolution techniques to the input images which may result in visual artifacts, or simply training one model for each resolution, which is impractical in many realistic applications. To address the above issues, this paper proposes a novel algorithm called RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. The proposed network is able to learn the 3D body shape and pose across different resolutions with a single model. The self-supervision loss encourages scale-consistency of the output, and the contrastive learning scheme enforces scale-consistency of the deep features. We show that both these new training losses provide robustness when learning 3D shape and pose in a weakly-supervised manner. Extensive experiments demonstrate that the RSC-Net can achieve consistently better results than the state-of-the-art methods for challenging low-resolution images.

READ FULL TEXT

page 2

page 11

page 14

research
10/03/2018

SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation

Recent techniques in self-supervised monocular depth estimation are appr...
research
08/20/2019

Deep High-Resolution Representation Learning for Visual Recognition

High-resolution representations are essential for position-sensitive vis...
research
07/16/2020

Human Pose Estimation on Privacy-Preserving Low-Resolution Depth Images

Human pose estimation (HPE) is a key building block for developing AI-ba...
research
03/08/2020

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

The primary aim of single-image super-resolution is to construct a high-...
research
07/07/2021

FasterPose: A Faster Simple Baseline for Human Pose Estimation

The performance of human pose estimation depends on the spatial accuracy...
research
11/24/2021

LDP-Net: An Unsupervised Pansharpening Network Based on Learnable Degradation Processes

Pansharpening in remote sensing image aims at acquiring a high-resolutio...
research
10/21/2022

FIND: An Unsupervised Implicit 3D Model of Articulated Human Feet

In this paper we present a high fidelity and articulated 3D human foot m...

Please sign up or login with your details

Forgot password? Click here to reset