CheapET-3: Cost-Efficient Use of Remote DNN Models

08/24/2022
by   Michael Weiss, et al.
0

On complex problems, state of the art prediction accuracy of Deep Neural Networks (DNN) can be achieved using very large-scale models, consisting of billions of parameters. Such models can only be run on dedicated servers, typically provided by a 3rd party service, which leads to a substantial monetary cost for every prediction. We propose a new software architecture for client-side applications, where a small local DNN is used alongside a remote large-scale model, aiming to make easy predictions locally at negligible monetary cost, while still leveraging the benefits of a large model for challenging inputs. In a proof of concept we reduce prediction cost by up to 50

READ FULL TEXT

page 1

page 2

page 3

research
04/05/2023

Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks

Recent decades have seen the rise of large-scale Deep Neural Networks (D...
research
07/03/2020

CacheNet: A Model Caching Framework for Deep Learning Inference on the Edge

The success of deep neural networks (DNN) in machine perception applicat...
research
06/03/2017

IDK Cascades: Fast Deep Learning by Learning not to Overthink

Advances in deep learning have led to substantial increases in predictio...
research
02/05/2019

Stabilizing Inputs to Approximated Nonlinear Functions for Inference with Homomorphic Encryption in Deep Neural Networks

Leveled Homomorphic Encryption (LHE) offers a potential solution that co...
research
10/09/2019

Deep neural network for pier scour prediction

With the advancement in computing power over last decades, deep neural n...
research
04/06/2021

Enabling Inference Privacy with Adaptive Noise Injection

User-facing software services are becoming increasingly reliant on remot...
research
07/01/2021

VeriDL: Integrity Verification of Outsourced Deep Learning Services (Extended Version)

Deep neural networks (DNNs) are prominent due to their superior performa...

Please sign up or login with your details

Forgot password? Click here to reset