Temporal Action Localization with Variance-Aware Networks

08/25/2020
by   Ting-Ting Xie, et al.
0

This work addresses the problem of temporal action localization with Variance-Aware Networks (VAN), i.e., DNNs that use second-order statistics in the input and/or the output of regression tasks. We first propose a network (VANp) that when presented with the second-order statistics of the input, i.e., each sample has a mean and a variance, it propagates the mean and the variance throughout the network to deliver outputs with second order statistics. In this framework, both the input and the output could be interpreted as Gaussians. To do so, we derive differentiable analytic solutions, or reasonable approximations, to propagate across commonly used NN layers. To train the network, we define a differentiable loss based on the KL-divergence between the predicted Gaussian and a Gaussian around the ground truth action borders, and use standard back-propagation. Importantly, the variances propagation in VANp does not require any additional parameters, and during testing, does not require any additional computations either. In action localization, the means and the variances of the input are computed at pooling operations, that are typically used to bring arbitrarily long videos to a vector with fixed dimensions. Second, we propose two alternative formulations that augment the first (respectively, the last) layer of a regression network with additional parameters so as to take in the input (respectively, predict in the output) both means and variances. Results in the action localization problem show that the incorporation of second order statistics improves over the baseline network, and that VANp surpasses the accuracy of virtually all other two-stage networks without involving any additional parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/25/2020

Boundary Uncertainty in a Single-Stage Temporal Action Localization Network

In this paper, we address the problem of temporal action localization wi...
research
11/16/2021

Second-order statistics of fermionic Gaussian states

We study the statistical behavior of entanglement in quantum bipartite s...
research
11/09/2018

Second order Stein: SURE for SURE and other applications in high-dimensional inference

Stein's formula states that a random variable of the form z^ f(z) - div ...
research
01/17/2022

A Theory of Second-Order Wireless Network Optimization and Its Application on AoI

This paper introduces a new theoretical framework for optimizing second-...
research
11/29/2020

Learning Affinity-Aware Upsampling for Deep Image Matting

We show that learning affinity in upsampling provides an effective and e...
research
05/21/2019

Robustness Against Outliers For Deep Neural Networks By Gradient Conjugate Priors

We analyze a new robust method for the reconstruction of probability dis...
research
09/26/2020

A light-weight method to foster the (Grad)CAM interpretability and explainability of classification networks

We consider a light-weight method which allows to improve the explainabi...

Please sign up or login with your details

Forgot password? Click here to reset