Don't forget, there is more than forgetting: new metrics for Continual Learning

10/31/2018
by   Natalia Díaz Rodríguez, et al.
0

Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and forward knowledge transfer, memory overhead as well as computational efficiency. Drawing inspiration from the standard Multi-Attribute Value Theory (MAVT) we further propose to fuse these metrics into a single score for ranking purposes and we evaluate our proposal with five continual learning strategies on the iCIFAR-100 continual learning benchmark.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/26/2017

Gradient Episodic Memory for Continual Learning

One major obstacle towards AI is the poor ability of models to solve new...
research
06/29/2019

Continual Learning for Robotics

Continual learning (CL) is a particular machine learning paradigm where ...
research
02/14/2022

Continual Learning from Demonstration of Robotic Skills

Methods for teaching motion skills to robots focus on training for a sin...
research
09/01/2023

New metrics for analyzing continual learners

Deep neural networks have shown remarkable performance when trained on i...
research
11/28/2022

Learning to Learn: How to Continuously Teach Humans and Machines

Our education system comprises a series of curricula. For example, when ...
research
07/08/2019

Fine-Grained Continual Learning

Robotic vision is a field where continual learning can play a significan...
research
01/31/2023

A Comprehensive Survey of Continual Learning: Theory, Method and Application

To cope with real-world dynamics, an intelligent agent needs to incremen...

Please sign up or login with your details

Forgot password? Click here to reset