Privileged Knowledge Distillation for Online Action Detection

11/18/2020
by   Peisen Zhao, et al.
0

Online Action Detection (OAD) in videos is proposed as a per-frame labeling task to address the real-time prediction tasks that can only obtain the previous and current video frames. This paper presents a novel learning-with-privileged based framework for online action detection where the future frames only observable at the training stages are considered as a form of privileged information. Knowledge distillation is employed to transfer the privileged information from the offline teacher to the online student. We note that this setting is different from conventional KD because the difference between the teacher and student models mostly lies in input data rather than the network architecture. We propose Privileged Knowledge Distillation (PKD) which (i) schedules a curriculum learning procedure and (ii) inserts auxiliary nodes to the student model, both for shrinking the information gap and improving learning performance. Compared to other OAD methods that explicitly predict future frames, our approach avoids learning unpredictable unnecessary yet inconsistent visual contents and achieves state-of-the-art accuracy on two popular OAD benchmarks, TVSeries and THUMOS14.

READ FULL TEXT
research
02/12/2021

Learning Student-Friendly Teacher Networks for Knowledge Distillation

We propose a novel knowledge distillation approach to facilitate the tra...
research
01/21/2021

Bridging the gap between Human Action Recognition and Online Action Detection

Action recognition, early prediction, and online action detection are co...
research
03/09/2022

How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting

Accurate prediction of future human positions is an essential task for m...
research
01/23/2022

ASCNet: Action Semantic Consistent Learning of Arbitrary Progress Levels for Early Action Prediction

Early action prediction aims to recognize human actions from only a part...
research
06/29/2023

Streaming egocentric action anticipation: An evaluation scheme and approach

Egocentric action anticipation aims to predict the future actions the ca...
research
07/10/2022

1st Place Solution to the EPIC-Kitchens Action Anticipation Challenge 2022

In this report, we describe the technical details of our submission to t...
research
05/12/2018

I Have Seen Enough: A Teacher Student Network for Video Classification Using Fewer Frames

Over the past few years, various tasks involving videos such as classifi...

Please sign up or login with your details

Forgot password? Click here to reset