Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection

03/29/2020
by   Jie Chen, et al.
20

We address weakly-supervised video actor-action segmentation (VAAS), which extends general video object segmentation (VOS) to additionally consider action labels of the actors. The most successful methods on VOS synthesize a pool of pseudo-annotations (PAs) and then refine them iteratively. However, they face challenges as to how to select from a massive amount of PAs high-quality ones, how to set an appropriate stop condition for weakly-supervised training, and how to initialize PAs pertaining to VAAS. To overcome these challenges, we propose a general Weakly-Supervised framework with a Wise Selection of training samples and model evaluation criterion (WS^2). Instead of blindly trusting quality-inconsistent PAs, WS^2 employs a learning-based selection to select effective PAs and a novel region integrity criterion as a stopping condition for weakly-supervised training. In addition, a 3D-Conv GCAM is devised to adapt to the VAAS task. Extensive experiments show that WS^2 achieves state-of-the-art performance on both weakly-supervised VOS and VAAS tasks and is on par with the best fully-supervised method on VAAS.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 8

research
04/05/2019

Weakly Supervised Action Segmentation Using Mutual Consistency

Action segmentation is the task of predicting the actions in each frame ...
research
05/19/2020

On Evaluating Weakly Supervised Action Segmentation Methods

Action segmentation is the task of temporally segmenting every frame of ...
research
03/31/2020

SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentation

Temporal action segmentation is a topic of increasing interest, however,...
research
01/14/2022

Transformers in Action: Weakly Supervised Action Segmentation

The video action segmentation task is regularly explored under weaker fo...
research
06/18/2023

Weakly Supervised Regression with Interval Targets

This paper investigates an interesting weakly supervised regression sett...
research
08/10/2018

Weakly-Supervised Attention and Relation Learning for Facial Action Unit Detection

Attention mechanism has recently attracted increasing attentions in the ...
research
09/03/2023

Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning

As advanced image manipulation techniques emerge, detecting the manipula...

Please sign up or login with your details

Forgot password? Click here to reset