Accretionary Learning with Deep Neural Networks

by   Xinyu Wei, et al.

One of the fundamental limitations of Deep Neural Networks (DNN) is its inability to acquire and accumulate new cognitive capabilities. When some new data appears, such as new object classes that are not in the prescribed set of objects being recognized, a conventional DNN would not be able to recognize them due to the fundamental formulation that it takes. The current solution is typically to re-design and re-learn the entire network, perhaps with a new configuration, from a newly expanded dataset to accommodate new knowledge. This process is quite different from that of a human learner. In this paper, we propose a new learning method named Accretionary Learning (AL) to emulate human learning, in that the set of objects to be recognized may not be pre-specified. The corresponding learning structure is modularized, which can dynamically expand to register and use new knowledge. During accretionary learning, the learning process does not require the system to be totally re-designed and re-trained as the set of objects grows in size. The proposed DNN structure does not forget previous knowledge when learning to recognize new data classes. We show that the new structure and the design methodology lead to a system that can grow to cope with increased cognitive complexity while providing stable and superior overall performance.


page 1

page 2

page 3

page 4


Bayesian Incremental Learning for Deep Neural Networks

In industrial machine learning pipelines, data often arrive in parts. Pa...

End-to-End Incremental Learning

Although deep learning approaches have stood out in recent years due to ...

Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks

Deep neural networks (DNNs) have become a widely deployed model for nume...

Abstract Learning via Demodulation in a Deep Neural Network

Inspired by the brain, deep neural networks (DNN) are thought to learn a...

Stable Architectures for Deep Neural Networks

Deep neural networks have become invaluable tools for supervised machine...

Deep Learning Approach for Very Similar Objects Recognition Application on Chihuahua and Muffin Problem

We address the problem to tackle the very similar objects like Chihuahua...

3D_DEN: Open-ended 3D Object Recognition using Dynamically Expandable Networks

Service robots, in general, have to work independently and adapt to the ...

Please sign up or login with your details

Forgot password? Click here to reset