Projected Subnetworks Scale Adaptation

01/27/2023
by   Siddhartha Datta, et al.
0

Large models support great zero-shot and few-shot capabilities. However, updating these models on new tasks can break performance on previous seen tasks and their zero/few-shot unseen tasks. Our work explores how to update zero/few-shot learners such that they can maintain performance on seen/unseen tasks of previous tasks as well as new tasks. By manipulating the parameter updates of a gradient-based meta learner as the projected task-specific subnetworks, we show improvements for large models to retain seen and zero/few shot task performance in online settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/24/2022

Exploring Euphemism Detection in Few-Shot and Zero-Shot Settings

This work builds upon the Euphemism Detection Shared Task proposed in th...
research
01/18/2022

ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization

We propose a multitask pretraining approach ZeroPrompt for zero-shot gen...
research
04/08/2022

Canonical Mean Filter for Almost Zero-Shot Multi-Task classification

The support set is a key to providing conditional prior for fast adaptio...
research
06/09/2022

On the Generalization and Adaption Performance of Causal Models

Learning models that offer robust out-of-distribution generalization and...
research
03/04/2019

Zero-Shot Task Transfer

In this work, we present a novel meta-learning algorithm, i.e. TTNet, th...
research
05/24/2023

A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classification

In recent years, large language models (LLMs) have achieved strong perfo...
research
03/01/2021

Performance Variability in Zero-Shot Classification

Zero-shot classification (ZSC) is the task of learning predictors for cl...

Please sign up or login with your details

Forgot password? Click here to reset