ODIN: Automated Drift Detection and Recovery in Video Analytics

09/09/2020
by   Abhijit Suprem, et al.
2

Recent advances in computer vision have led to a resurgence of interest in visual data analytics. Researchers are developing systems for effectively and efficiently analyzing visual data at scale. A significant challenge that these systems encounter lies in the drift in real-world visual data. For instance, a model for self-driving vehicles that is not trained on images containing snow does not work well when it encounters them in practice. This drift phenomenon limits the accuracy of models employed for visual data analytics. In this paper, we present a visual data analytics system, called ODIN, that automatically detects and recovers from drift. ODIN uses adversarial autoencoders to learn the distribution of high-dimensional images. We present an unsupervised algorithm for detecting drift by comparing the distributions of the given data against that of previously seen data. When ODIN detects drift, it invokes a drift recovery algorithm to deploy specialized models tailored towards the novel data points. These specialized models outperform their non-specialized counterpart on accuracy, performance, and memory footprint. Lastly, we present a model selection algorithm for picking an ensemble of best-fit specialized models to process a given input. We evaluate the efficacy and efficiency of ODIN on high-resolution dashboard camera videos captured under diverse environments from the Berkeley DeepDrive dataset. We demonstrate that ODIN's models deliver 6x higher throughput, 2x higher accuracy, and 6x smaller memory footprint compared to a baseline system without automated drift detection and recovery.

READ FULL TEXT

Authors

page 3

page 5

07/28/2020

Diagnosing Concept Drift with Visual Analytics

Concept drift is a phenomenon in which the distribution of a data stream...
03/02/2020

Finding Interesting Frames in Deep Video Analytics: a Top-K Approach

Recently, the impressive accuracy of deep neural networks (DNNs) has cre...
12/19/2020

Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers

Video analytics applications use edge compute servers for the analytics ...
04/07/2020

Adversarial Validation Approach to Concept Drift Problem in Automated Machine Learning Systems

In automated machine learning systems, concept drift in input data is on...
07/01/2021

Unsupervised Model Drift Estimation with Batch Normalization Statistics for Dataset Shift Detection and Model Selection

While many real-world data streams imply that they change frequently in ...
02/16/2021

THIA: Accelerating Video Analytics using Early Inference and Fine-Grained Query Planning

To efficiently process visual data at scale, researchers have proposed t...
12/16/2021

Effective prevention of semantic drift as angular distance in memory-less continual deep neural networks

Lifelong machine learning or continual learning models attempt to learn ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.