DeepAI AI Chat
Log In Sign Up

Characterizing and Taming Model Instability Across Edge Devices

by   Eyal Cidon, et al.
Stanford University

The same machine learning model running on different edge devices may produce highly-divergent outputs on a nearly-identical input. Possible reasons for the divergence include differences in the device sensors, the device's signal processing hardware and software, and its operating system and processors. This paper presents the first methodical characterization of the variations in model prediction across real-world mobile devices. We demonstrate that accuracy is not a useful metric to characterize prediction divergence, and introduce a new metric, instability, which captures this variation. We characterize different sources for instability, and show that differences in compression formats and image signal processing account for significant instability in object classification models. Notably, in our experiments, 14-17 divergent classifications across one or more phone models. We evaluate three different techniques for reducing instability. In particular, we adapt prior work on making models robust to noise in order to fine-tune models to be robust to variations across edge devices. We demonstrate our fine-tuning techniques reduce instability by 75


page 3

page 6

page 9


Efficient Fine-Tuning of BERT Models on the Edge

Resource-constrained devices are increasingly the deployment targets of ...

An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learning

To enable the pre-trained models to be fine-tuned with local data on edg...

Edge Deep Learning Model Protection via Neuron Authorization

With the development of deep learning processors and accelerators, deep ...

On-Device Model Fine-Tuning with Label Correction in Recommender Systems

To meet the practical requirements of low latency, low cost, and good pr...

A Tale of Two Models: Constructing Evasive Attacks on Edge Models

Full-precision deep learning models are typically too large or costly to...

What Happens on the Edge, Stays on the Edge: Toward Compressive Deep Learning

Machine learning at the edge offers great benefits such as increased pri...

DRAWNAPART: A Device Identification Technique based on Remote GPU Fingerprinting

Browser fingerprinting aims to identify users or their devices, through ...