DeepAI AI Chat
Log In Sign Up

Unsupervised Motion Representation Enhanced Network for Action Recognition

by   Xiaohang Yang, et al.

Learning reliable motion representation between consecutive frames, such as optical flow, has proven to have great promotion to video understanding. However, the TV-L1 method, an effective optical flow solver, is time-consuming and expensive in storage for caching the extracted optical flow. To fill the gap, we propose UF-TSN, a novel end-to-end action recognition approach enhanced with an embedded lightweight unsupervised optical flow estimator. UF-TSN estimates motion cues from adjacent frames in a coarse-to-fine manner and focuses on small displacement for each level by extracting pyramid of feature and warping one to the other according to the estimated flow of the last level. Due to the lack of labeled motion for action datasets, we constrain the flow prediction with multi-scale photometric consistency and edge-aware smoothness. Compared with state-of-the-art unsupervised motion representation learning methods, our model achieves better accuracy while maintaining efficiency, which is competitive with some supervised or more complicated approaches.


On the Integration of Optical Flow and Action Recognition

Most of the top performing action recognition methods use optical flow a...

End-to-End Learning of Motion Representation for Video Understanding

Despite the recent success of end-to-end learned representations, hand-c...

Unsupervised Learning for Optical Flow Estimation Using Pyramid Convolution LSTM

Most of current Convolution Neural Network (CNN) based methods for optic...

DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition

Motion has shown to be useful for video understanding, where motion is t...

Unsupervised Optical Flow Using Cost Function Unrolling

Analyzing motion between two consecutive images is one of the fundamenta...

Real-time Action Recognition with Enhanced Motion Vector CNNs

The deep two-stream architecture exhibited excellent performance on vide...

Exploiting Inter-Frame Regional Correlation for Efficient Action Recognition

Temporal feature extraction is an important issue in video-based action ...