Combining Modular Skills in Multitask Learning

02/28/2022
by   Edoardo M. Ponti, et al.
9

A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume that each task is associated with a subset of latent discrete skills from a (potentially small) inventory. In turn, skills correspond to parameter-efficient (sparse / low-rank) model parameterisations. By jointly learning these and a task-skill allocation matrix, the network for each task is instantiated as the average of the parameters of active skills. To favour non-trivial soft partitions of skills across tasks, we experiment with a series of inductive biases, such as an Indian Buffet Process prior and a two-speed learning rate. We evaluate our latent-skill model on two main settings: 1) multitask reinforcement learning for grounded instruction following on 8 levels of the BabyAI platform; and 2) few-shot adaptation of pre-trained text-to-text generative models on CrossFit, a benchmark comprising 160 NLP tasks. We find that the modular design of a network significantly increases sample efficiency in reinforcement learning and few-shot generalisation in supervised learning, compared to baselines with fully shared, task-specific, or conditionally generated parameters where knowledge is entangled across tasks. In addition, we show how discrete skills help interpretability, as they yield an explicit hierarchy of tasks.

READ FULL TEXT

page 9

page 15

research
02/21/2020

Modelling Latent Skills for Multitask Language Generation

We present a generative model for multitask conditional language generat...
research
06/28/2023

SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills

Traditional multitask learning methods basically can only exploit common...
research
06/26/2021

Discovering Generalizable Skills via Automated Generation of Diverse Tasks

The learning efficiency and generalization ability of an intelligent age...
research
10/20/2021

Hierarchical Skills for Efficient Exploration

In reinforcement learning, pre-trained low-level skills have the potenti...
research
12/08/2022

Learning Options via Compression

Identifying statistical regularities in solutions to some tasks in multi...
research
06/21/2019

Disentangled Skill Embeddings for Reinforcement Learning

We propose a novel framework for multi-task reinforcement learning (MTRL...

Please sign up or login with your details

Forgot password? Click here to reset