Serving Deep Learning Models with Deduplication from Relational Databases

01/25/2022
by   Lixi Zhou, et al.
0

There are significant benefits to serve deep learning models from relational databases. First, features extracted from databases do not need to be transferred to any decoupled deep learning systems for inferences, and thus the system management overhead can be significantly reduced. Second, in a relational database, data management along the storage hierarchy is fully integrated with query processing, and thus it can continue model serving even if the working set size exceeds the available memory. Applying model deduplication can greatly reduce the storage space, memory footprint, cache misses, and inference latency. However, existing data deduplication techniques are not applicable to the deep learning model serving applications in relational databases. They do not consider the impacts on model inference accuracy as well as the inconsistency between tensor blocks and database pages. This work proposed synergistic storage optimization techniques for duplication detection, page packing, and caching, to enhance database systems for model serving. We implemented the proposed approach in netsDB, an object-oriented relational database. Evaluation results show that our proposed techniques significantly improved the storage efficiency and the model inference latency, and serving models from relational databases outperformed existing deep learning frameworks when the working set size exceeds available memory.

READ FULL TEXT

page 3

page 5

page 8

page 9

page 10

page 11

page 13

page 14

research
03/21/2019

Deep Models for Relational Databases

Due to its extensive use in databases, the relational model is ubiquitou...
research
02/20/2015

An Approach For Transforming of Relational Databases to OWL Ontology

Rapid growth of documents, web pages, and other types of text content is...
research
11/04/2020

Understanding Capacity-Driven Scale-Out Neural Recommendation Inference

Deep learning recommendation models have grown to the terabyte scale. Tr...
research
04/30/2021

BERT Meets Relational DB: Contextual Representations of Relational Databases

In this paper, we address the problem of learning low dimension represen...
research
11/15/2019

Learning Models over Relational Data: A Brief Tutorial

This tutorial overviews the state of the art in learning models over rel...
research
11/18/2016

ModelHub: Towards Unified Data and Lifecycle Management for Deep Learning

Deep learning has improved state-of-the-art results in many important fi...
research
11/17/2019

Caching Techniques to Improve Latency in Serverless Architectures

Serverless computing has gained a significant traction in recent times b...

Please sign up or login with your details

Forgot password? Click here to reset