Lifelong Teacher-Student Network Learning

07/09/2021
by   Fei Ye, et al.
6

A unique cognitive capability of humans consists in their ability to acquire new knowledge and skills from a sequence of experiences. Meanwhile, artificial intelligence systems are good at learning only the last given task without being able to remember the databases learnt in the past. We propose a novel lifelong learning methodology by employing a Teacher-Student network framework. While the Student module is trained with a new given database, the Teacher module would remind the Student about the information learnt in the past. The Teacher, implemented by a Generative Adversarial Network (GAN), is trained to preserve and replay past knowledge corresponding to the probabilistic representations of previously learn databases. Meanwhile, the Student module is implemented by a Variational Autoencoder (VAE) which infers its latent variable representation from both the output of the Teacher module as well as from the newly available database. Moreover, the Student module is trained to capture both continuous and discrete underlying data representations across different domains. The proposed lifelong learning framework is applied in supervised, semi-supervised and unsupervised training. The code is available : <https://github.com/dtuzi123/Lifelong-Teacher-Student-Network-Learning>

READ FULL TEXT

page 9

page 10

page 11

page 12

page 14

page 15

research
03/12/2019

Semi-Supervised Self-Taught Deep Learning for Finger Bones Segmentation

Segmentation stands at the forefront of many high-level vision tasks. In...
research
05/27/2017

Lifelong Generative Modeling

Lifelong learning is the problem of learning multiple consecutive tasks ...
research
05/23/2023

NORM: Knowledge Distillation via N-to-One Representation Matching

Existing feature distillation methods commonly adopt the One-to-one Repr...
research
09/05/2017

Knowledge Transfer Between Artificial Intelligence Systems

We consider the fundamental question: how a legacy "student" Artificial ...
research
07/27/2018

One-Shot Optimal Topology Generation through Theory-Driven Machine Learning

We introduce a theory-driven mechanism for learning a neural network mod...
research
12/15/2021

Lifelong Generative Modelling Using Dynamic Expansion Graph Model

Variational Autoencoders (VAEs) suffer from degenerated performance, whe...
research
06/21/2018

Learning Cognitive Models using Neural Networks

A cognitive model of human learning provides information about skills a ...

Please sign up or login with your details

Forgot password? Click here to reset