Lifelong Learning of Compositional Structures

by   Jorge A. Mendez, et al.

A hallmark of human intelligence is the ability to construct self-contained chunks of knowledge and adequately reuse them in novel combinations for solving different yet structurally related problems. Learning such compositional structures has been a significant challenge for artificial systems, due to the combinatorial nature of the underlying search problem. To date, research into compositional learning has largely proceeded separately from work on lifelong or continual learning. We integrate these two lines of work to present a general-purpose framework for lifelong learning of compositional structures that can be used for solving a stream of related tasks. Our framework separates the learning process into two broad stages: learning how to best combine existing components in order to assimilate a novel problem, and learning how to adapt the set of existing components to accommodate the new problem. This separation explicitly handles the trade-off between the stability required to remember how to solve earlier tasks and the flexibility required to solve new tasks, as we show empirically in an extensive evaluation.



There are no comments yet.


page 3

page 13

page 16


Visually Grounded Continual Learning of Compositional Semantics

Children's language acquisition from the visual world is a real-world ex...

Concepts, Properties and an Approach for Compositional Generalization

Compositional generalization is the capacity to recognize and imagine a ...

Continual Lifelong Learning in Natural Language Processing: A Survey

Continual learning (CL) aims to enable information systems to learn from...

Continual learning: A comparative study on how to defy forgetting in classification tasks

Artificial neural networks thrive in solving the classification problem ...

CLeaR: An Adaptive Continual Learning Framework for Regression Tasks

Catastrophic forgetting means that a trained neural network model gradua...

Learning Compositional Neural Information Fusion for Human Parsing

This work proposes to combine neural networks with the compositional hie...

Measuring Compositionality in Representation Learning

Many machine learning algorithms represent input data with vector embedd...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.