Multiview Supervision By Registration

11/27/2018
by   Yilun Zhang, et al.
0

This paper presents a semi-supervised learning framework to train a keypoint pose detector using multiview image streams given the limited number of labeled data (typically <4 multiview geometry and visual tracking to provide three types of supervisionary signals for the unlabeled data: (1) pose detection in one view can be used to supervise that of the other view as they must satisfy the epipolar constraint; (2) pose detection must be temporally coherent in accordance with its optical flow; (3) the occluded keypoint from one view must be consistently invisible from the near views. We formulate the theory of multiview supervision by registration and design a new end-to-end neural network that integrates these supervisionary signals in a differentiable fashion to incorporate the large unlabeled data in pose detector training. The key innovation of the network is the ability to reason about the visibility/occlusion, which is indicative of the degenerate case of detection and tracking. Our resulting pose detector shows considerable outperformance comparing the state-of-the-art pose detectors in terms of accuracy (keypoint detection) and precision (3D reconstruction). We validate our approach with challenging realworld data including the pose detection of non-human species such as monkeys and dogs.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 9

research
05/31/2018

MONET: Multiview Semi-supervised Keypoint via Epipolar Divergence

This paper presents MONET---an end-to-end semi-supervised learning frame...
research
01/25/2021

Supervision by Registration and Triangulation for Landmark Detection

We present Supervision by Registration and Triangulation (SRT), an unsup...
research
09/20/2021

Semi-supervised Dense Keypointsusing Unlabeled Multiview Images

This paper presents a new end-to-end semi-supervised framework to learn ...
research
07/03/2018

Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors

In this paper, we present supervision-by-registration, an unsupervised a...
research
12/04/2018

Multiview Cross-supervision for Semantic Segmentation

This paper presents a semi-supervised learning framework for a customize...
research
02/03/2019

Semi-Supervised Learning Detector for MU-MIMO Systems with One-bit ADCs

We study an uplink multiuser multiple-input multiple-output (MU-MIMO) sy...
research
02/03/2020

Towards High Performance Human Keypoint Detection

Human keypoint detection from a single image is very challenging due to ...

Please sign up or login with your details

Forgot password? Click here to reset