Expert Gate: Lifelong Learning with a Network of Experts

11/18/2016
by   Rahaf Aljundi, et al.
0

In this paper we introduce a model of lifelong learning, based on a Network of Experts. New tasks / experts are learned and added to the model sequentially, building on what was learned before. To ensure scalability of this process,data from previous tasks cannot be stored and hence is not available when learning a new task. A critical issue in such context, not addressed in the literature so far, relates to the decision which expert to deploy at test time. We introduce a set of gating autoencoders that learn a representation for the task at hand, and, at test time, automatically forward the test sample to the relevant expert. This also brings memory efficiency as only one expert network has to be loaded into memory at any given time. Further, the autoencoders inherently capture the relatedness of one task to another, based on which the most relevant prior model to be used for training a new expert, with finetuning or learning without-forgetting, can be selected. We evaluate our method on image classification and video prediction problems.

READ FULL TEXT
research
09/15/2022

Test-Time Training with Masked Autoencoders

Test-time training adapts to a new test distribution on the fly by optim...
research
06/26/2020

Supermasks in Superposition

We present the Supermasks in Superposition (SupSup) model, capable of se...
research
07/09/2021

Lifelong Mixture of Variational Autoencoders

In this paper, we propose an end-to-end lifelong learning mixture of exp...
research
02/28/2023

Improving Expert Specialization in Mixture of Experts

Mixture of experts (MoE), introduced over 20 years ago, is the simplest ...
research
04/06/2017

Encoder Based Lifelong Learning

This paper introduces a new lifelong learning solution where a single mo...
research
08/11/2021

DEMix Layers: Disentangling Domains for Modular Language Modeling

We introduce a new domain expert mixture (DEMix) layer that enables cond...
research
12/06/2021

Anchoring to Exemplars for Training Mixture-of-Expert Cell Embeddings

Analyzing the morphology of cells in microscopy images can provide insig...

Please sign up or login with your details

Forgot password? Click here to reset