DeepAI AI Chat
Log In Sign Up

Exploiting a Zoo of Checkpoints for Unseen Tasks

11/05/2021
by   Jiaji Huang, et al.
Purdue University
Baidu, Inc.
0

There are so many models in the literature that it is difficult for practitioners to decide which combinations are likely to be effective for a new task. This paper attempts to address this question by capturing relationships among checkpoints published on the web. We model the space of tasks as a Gaussian process. The covariance can be estimated from checkpoints and unlabeled probing data. With the Gaussian process, we can identify representative checkpoints by a maximum mutual information criterion. This objective is submodular. A greedy method identifies representatives that are likely to "cover" the task space. These representatives generalize to new tasks with superior performance. Empirical evidence is provided for applications from both computational linguistics as well as computer vision.

READ FULL TEXT
07/30/2021

Active Learning in Gaussian Process State Space Model

We investigate active learning in Gaussian Process state-space models (G...
10/21/2019

Empirical Process of Multivariate Gaussian under General Dependence

This paper explores certain kinds of empirical process with respect to t...
10/21/2019

Empirical Process of Multivariate Gaussian under General Dependency

This paper explores certain kinds of empirical process with respect to t...
10/04/2016

Model Selection for Gaussian Process Regression by Approximation Set Coding

Gaussian processes are powerful, yet analytically tractable models for s...
09/17/2012

Submodularity in Batch Active Learning and Survey Problems on Gaussian Random Fields

Many real-world datasets can be represented in the form of a graph whose...