DeepAI
Log In Sign Up

SiamParseNet: Joint Body Parsing and Label Propagation in Infant Movement Videos

07/16/2020
by   Haomiao Ni, et al.
6

General movement assessment (GMA) of infant movement videos (IMVs) is an effective method for the early detection of cerebral palsy (CP) in infants. Automated body parsing is a crucial step towards computer-aided GMA, in which infant body parts are segmented and tracked over time for movement analysis. However, acquiring fully annotated data for video-based body parsing is particularly expensive due to the large number of frames in IMVs. In this paper, we propose a semi-supervised body parsing model, termed SiamParseNet (SPN), to jointly learn single frame body parsing and label propagation between frames in a semi-supervised fashion. The Siamese-structured SPN consists of a shared feature encoder, followed by two separate branches: one for intra-frame body parts segmentation, and one for inter-frame label propagation. The two branches are trained jointly, taking pairs of frames from the same videos as their input. An adaptive training process is proposed that alternates training modes between using input pairs of only labeled frames and using inputs of both labeled and unlabeled frames. During testing, we employ a multi-source inference mechanism, where the final result for a test frame is either obtained via the segmentation branch or via propagation from a nearby key frame. We conduct extensive experiments on a partially-labeled IMV dataset where SPN outperforms all prior arts, demonstrating the effectiveness of our proposed method.

READ FULL TEXT

page 2

page 8

10/14/2022

Semi-supervised Body Parsing and Pose Estimation for Enhancing Infant General Movement Assessment

General movement assessment (GMA) of infant movement videos (IMVs) is an...
08/02/2018

Adaptive Temporal Encoding Network for Video Instance-level Human Parsing

Beyond the existing single-person and multiple-person human parsing task...
07/06/2020

Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video

Performing low hertz labeling for surgical videos at intervals can great...
10/02/2020

Semantics through Time: Semi-supervised Segmentation of Aerial Videos with Iterative Label Propagation

Semantic segmentation is a crucial task for robot navigation and safety....
09/28/2021

Warp-Refine Propagation: Semi-Supervised Auto-labeling via Cycle-consistency

Deep learning models for semantic segmentation rely on expensive, large-...
05/27/2018

Dual Swap Disentangling

Learning interpretable disentangled representations is a crucial yet cha...
03/25/2022

Semi-supervised and Deep learning Frameworks for Video Classification and Key-frame Identification

Automating video-based data and machine learning pipelines poses several...

Code Repositories

MICCAI20_SiamParseNet

The implementation of our MICCAI20 paper "SiamParseNet: Joint Body Parsing and Label Propagation in Infant Movement Videos"


view repo