Context Recovery and Knowledge Retrieval: A Novel Two-Stream Framework for Video Anomaly Detection

09/07/2022
by   Congqi Cao, et al.
0

Video anomaly detection aims to find the events in a video that do not conform to the expected behavior. The prevalent methods mainly detect anomalies by snippet reconstruction or future frame prediction error. However, the error is highly dependent on the local context of the current snippet and lacks the understanding of normality. To address this issue, we propose to detect anomalous events not only by the local context, but also according to the consistency between the testing event and the knowledge about normality from the training data. Concretely, we propose a novel two-stream framework based on context recovery and knowledge retrieval, where the two streams can complement each other. For the context recovery stream, we propose a spatiotemporal U-Net which can fully utilize the motion information to predict the future frame. Furthermore, we propose a maximum local error mechanism to alleviate the problem of large recovery errors caused by complex foreground objects. For the knowledge retrieval stream, we propose an improved learnable locality-sensitive hashing, which optimizes hash functions via a Siamese network and a mutual difference loss. The knowledge about normality is encoded and stored in hash tables, and the distance between the testing event and the knowledge representation is used to reveal the probability of anomaly. Finally, we fuse the anomaly scores from the two streams to detect anomalies. Extensive experiments demonstrate the effectiveness and complementarity of the two streams, whereby the proposed two-stream framework achieves state-of-the-art performance on four datasets.

READ FULL TEXT

page 1

page 11

research
03/09/2023

Updated version: A Video Anomaly Detection Framework based on Appearance-Motion Semantics Representation Consistency

Video anomaly detection is an essential but challenging task. The preval...
research
11/15/2021

Learnable Locality-Sensitive Hashing for Video Anomaly Detection

Video anomaly detection (VAD) mainly refers to identifying anomalous eve...
research
11/19/2019

A Boost Strategy to the Generative Error Based Video Anomaly Detection Algorithms

The generation error (GE) based algorithms show excellent performances i...
research
06/17/2022

Multi-Contextual Predictions with Vision Transformer for Video Anomaly Detection

Video Anomaly Detection(VAD) has been traditionally tackled in two main ...
research
08/04/2021

Sensing Anomalies like Humans: A Hominine Framework to Detect Abnormal Events from Unlabeled Videos

Video anomaly detection (VAD) has constantly been a vital topic in video...
research
03/07/2023

Fast and Multi-aspect Mining of Complex Time-stamped Event Streams

Given a huge, online stream of time-evolving events with multiple attrib...
research
08/07/2018

Predicting Visual Context for Unsupervised Event Segmentation in Continuous Photo-streams

Segmenting video content into events provides semantic structures for in...

Please sign up or login with your details

Forgot password? Click here to reset