Temporal Bilinear Networks for Video Action Recognition

11/25/2018
by   Yanghao Li, et al.
0

Temporal modeling in videos is a fundamental yet challenging problem in computer vision. In this paper, we propose a novel Temporal Bilinear (TB) model to capture the temporal pairwise feature interactions between adjacent frames. Compared with some existing temporal methods which are limited in linear transformations, our TB model considers explicit quadratic bilinear transformations in the temporal domain for motion evolution and sequential relation modeling. We further leverage the factorized bilinear model in linear complexity and a bottleneck network design to build our TB blocks, which also constrains the parameters and computation cost. We consider two schemes in terms of the incorporation of TB blocks and the original 2D spatial convolutions, namely wide and deep Temporal Bilinear Networks (TBN). Finally, we perform experiments on several widely adopted datasets including Kinetics, UCF101 and HMDB51. The effectiveness of our TBNs is validated by comprehensive ablation analyses and comparisons with various state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/17/2016

Factorized Bilinear Models for Image Recognition

Although Deep Convolutional Neural Networks (CNNs) have liberated their ...
research
07/25/2020

Approximated Bilinear Modules for Temporal Modeling

We consider two less-emphasized temporal properties of video: 1. Tempora...
research
11/24/2017

Appearance-and-Relation Networks for Video Classification

Spatiotemporal feature learning in videos is a fundamental and difficult...
research
02/20/2020

Complete Endomorphisms in Computer Vision

Correspondences between k-tuples of points are key in multiple view geom...
research
12/05/2018

Local Temporal Bilinear Pooling for Fine-grained Action Parsing

Fine-grained temporal action parsing is important in many applications, ...
research
01/19/2021

Hybrid Trilinear and Bilinear Programming for Aligning Partially Overlapping Point Sets

Alignment methods which can handle partially overlapping point sets and ...
research
12/20/2017

An Order Preserving Bilinear Model for Person Detection in Multi-Modal Data

We propose a new order preserving bilinear framework that exploits low-r...

Please sign up or login with your details

Forgot password? Click here to reset