Online Continual Learning for Embedded Devices

03/21/2022
by   Tyler L. Hayes, et al.
0

Real-time on-device continual learning is needed for new applications such as home robots, user personalization on smartphones, and augmented/virtual reality headsets. However, this setting poses unique challenges: embedded devices have limited memory and compute capacity and conventional machine learning models suffer from catastrophic forgetting when updated on non-stationary data streams. While several online continual learning models have been developed, their effectiveness for embedded applications has not been rigorously studied. In this paper, we first identify criteria that online continual learners must meet to effectively perform real-time, on-device learning. We then study the efficacy of several online continual learning methods when used with mobile neural networks. We measure their performance, memory usage, compute requirements, and ability to generalize to out-of-domain inputs.

READ FULL TEXT

page 2

page 5

research
10/07/2019

Continual Learning in Neural Networks

Artificial neural networks have exceeded human-level performance in acco...
research
06/23/2023

Explainable Lifelong Stream Learning Based on "Glocal" Pairwise Fusion

Real-time on-device continual learning applications are used on mobile p...
research
05/05/2021

Continual Learning on the Edge with TensorFlow Lite

Deploying sophisticated deep learning models on embedded devices with th...
research
01/18/2022

Continual Learning for CTR Prediction: A Hybrid Approach

Click-through rate(CTR) prediction is a core task in cost-per-click(CPC)...
research
07/24/2023

Online Continual Learning in Keyword Spotting for Low-Resource Devices via Pooling High-Order Temporal Statistics

Keyword Spotting (KWS) models on embedded devices should adapt fast to n...
research
07/19/2023

Online Continual Learning for Robust Indoor Object Recognition

Vision systems mounted on home robots need to interact with unseen class...
research
03/17/2023

CoVIO: Online Continual Learning for Visual-Inertial Odometry

Visual odometry is a fundamental task for many applications on mobile de...

Please sign up or login with your details

Forgot password? Click here to reset