Benchmark of DNN Model Search at Deployment Time

06/01/2022
by   Lixi Zhou, et al.
0

Deep learning has become the most popular direction in machine learning and artificial intelligence. However, the preparation of training data, as well as model training, are often time-consuming and become the bottleneck of the end-to-end machine learning lifecycle. Reusing models for inferring a dataset can avoid the costs of retraining. However, when there are multiple candidate models, it is challenging to discover the right model for reuse. Although there exist a number of model sharing platforms such as ModelDB, TensorFlow Hub, PyTorch Hub, and DLHub, most of these systems require model uploaders to manually specify the details of each model and model downloaders to screen keyword search results for selecting a model. We are lacking a highly productive model search tool that selects models for deployment without the need for any manual inspection and/or labeled data from the target domain. This paper proposes multiple model search strategies including various similarity-based approaches and non-similarity-based approaches. We design, implement, and evaluate these approaches on multiple model inference scenarios, including activity recognition, image recognition, text classification, natural language processing, and entity matching. The experimental evaluation showed that our proposed asymmetric similarity-based measurement, adaptivity, outperformed symmetric similarity-based measurements and non-similarity-based measurements in most of the workloads.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/13/2020

It's the Best Only When It Fits You Most: Finding Related Models for Serving Based on Dynamic Locality Sensitive Hashing

In recent, deep learning has become the most popular direction in machin...
research
02/11/2022

Similarity learning for wells based on logging data

One of the first steps during the investigation of geological objects is...
research
06/23/2023

Combining Public Human Activity Recognition Datasets to Mitigate Labeled Data Scarcity

The use of supervised learning for Human Activity Recognition (HAR) on m...
research
11/29/2022

Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches

Text classification of unseen classes is a challenging Natural Language ...
research
06/08/2023

Neuro-Symbolic Approaches for Context-Aware Human Activity Recognition

Deep Learning models are a standard solution for sensor-based Human Acti...
research
06/14/2022

Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition

It is expensive and time-consuming to collect sufficient labeled data to...
research
01/10/2019

Automating the search for a patent's prior art with a full text similarity search

More than ever, technical inventions are the symbol of our society's adv...

Please sign up or login with your details

Forgot password? Click here to reset