Reinventing Data Stores for Video Analytics
share
We present a data store managing large videos for retrospective analytics. Our data store orchestrates video ingestion, storage, retrieval, and consumption. Towards resource efficiency, it takes the key opportunity of controlling the video formats along the video data path. We are challenged by i) the huge combinatorial space of video format knobs; ii) the complex impacts of these knobs and their high profiling cost; iii) optimizing for multiple resource types. To this end, our data store builds on a key idea called backward derivation of configuration: in the opposite direction along the video data path, the data store passes the video quantity and quality desired by analytics backward to retrieval, to storage, and to ingestion. In this process, our data store derives an optimal set of video formats, optimizing for different system resources in a progressive manner. Our data store automatically derives large, complex configurations consisting of hundreds of knobs. It streams video data from disks through decoder to operators and runs queries as fast as 362x of video realtime.
READ FULL TEXT