Llama: A Heterogeneous Serverless Framework for Auto-Tuning Video Analytics Pipelines

02/03/2021
by   Francisco Romero, et al.
0

The proliferation of camera-enabled devices and large video repositories has given rise to a diverse set of video analytics applications. The video pipelines for these applications are DAGs of operations that transform videos, process extracted metadata, and answer questions such as, "Is this intersection congested?" The latency and resource efficiency of pipelines can be optimized using configurable knobs for each operation such as the sampling rate, batch size, or type of hardware used. However, determining efficient configurations is challenging because (a) the configuration search space is exponentially large, and (b) the optimal configuration depends on the desired latency target and the input video contents that may exercise different paths in the DAG and produce different volumes of intermediate results. Existing video analytics and processing systems leave it to the users to manually configure operations and select hardware resources. Hence, we observe that they often execute inefficiently and fail to meet latency and cost targets. We present Llama: a heterogeneous and serverless framework for auto-tuning video pipelines. Llama optimizes the overall video pipeline latency by (a) dynamically calculating latency targets per-operation invocation, and (b) dynamically running a cost-based optimizer to determine efficient configurations that meet the target latency for each invocation. This makes the problem of auto-tuning large video pipelines tractable and allows us to handle input dependent behavior, conditional branches in the DAG, and execution variability. We describe the algorithms in Llama and evaluate it on a cloud platform using serverless CPU and GPU resources. We show that compared to state-of-the-art cluster and serverless video analytics and processing systems, Llama achieves 7.9x lower latency and 17.2x cost reduction on average.

READ FULL TEXT
research
06/29/2022

Turbo: Opportunistic Enhancement for Edge Video Analytics

Edge computing is being widely used for video analytics. To alleviate th...
research
02/11/2021

Effectively Testing System Configurations of Critical IoT Analytics Pipelines

The emergence of the Internet of Things has seen the introduction of num...
research
01/12/2023

HEP-BNN: A Framework for Finding Low-Latency Execution Configurations of BNNs on Heterogeneous Multiprocessor Platforms

Binarized Neural Networks (BNNs) significantly reduce the computation an...
research
04/22/2022

SCOPE: Safe Exploration for Dynamic Computer Systems Optimization

Modern computer systems need to execute under strict safety constraints ...
research
04/04/2023

MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations

Camera orientations (i.e., rotation and zoom) govern the content that a ...
research
10/03/2018

VStore: A Data Store for Analytics on Large Videos

We present VStore, a data store for supporting fast, resource-efficient ...
research
04/11/2022

Cello: Efficient Computer Systems Optimization with Predictive Early Termination and Censored Regression

Sample-efficient machine learning (SEML) has been widely applied to find...

Please sign up or login with your details

Forgot password? Click here to reset