Lifelong Machine Learning of Functionally Compositional Structures

by   Jorge A. Mendez, et al.

A hallmark of human intelligence is the ability to construct self-contained chunks of knowledge and reuse them in novel combinations for solving different problems. Learning such compositional structures has been a challenge for artificial systems, due to the underlying combinatorial search. To date, research into compositional learning has largely proceeded separately from work on lifelong or continual learning. This dissertation integrated these two lines of work to present a general-purpose framework for lifelong learning of functionally compositional structures. The framework separates the learning into two stages: learning how to combine existing components to assimilate a novel problem, and learning how to adapt the existing components to accommodate the new problem. This separation explicitly handles the trade-off between stability and flexibility. This dissertation instantiated the framework into various supervised and reinforcement learning (RL) algorithms. Supervised learning evaluations found that 1) compositional models improve lifelong learning of diverse tasks, 2) the multi-stage process permits lifelong learning of compositional knowledge, and 3) the components learned by the framework represent self-contained and reusable functions. Similar RL evaluations demonstrated that 1) algorithms under the framework accelerate the discovery of high-performing policies, and 2) these algorithms retain or improve performance on previously learned tasks. The dissertation extended one lifelong compositional RL algorithm to the nonstationary setting, where the task distribution varies over time, and found that modularity permits individually tracking changes to different elements in the environment. The final contribution of this dissertation was a new benchmark for compositional RL, which exposed that existing methods struggle to discover the compositional properties of the environment.


Lifelong Learning of Compositional Structures

A hallmark of human intelligence is the ability to construct self-contai...

Modular Lifelong Reinforcement Learning via Neural Composition

Humans commonly solve complex problems by decomposing them into easier s...

How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition

A major goal of artificial intelligence (AI) is to create an agent capab...

CompoSuite: A Compositional Reinforcement Learning Benchmark

We present CompoSuite, an open-source simulated robotic manipulation ben...

Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning

Offline reinforcement learning (RL) is a promising direction that allows...

Categorical semantics of compositional reinforcement learning

Reinforcement learning (RL) often requires decomposing a problem into su...

Meta-Referential Games to Learn Compositional Learning Behaviours

Human beings use compositionality to generalise from past experiences to...

Please sign up or login with your details

Forgot password? Click here to reset