Adaptive transfer learning

06/08/2021
by   Henry W J Reeve, et al.
13

In transfer learning, we wish to make inference about a target population when we have access to data both from the distribution itself, and from a different but related source distribution. We introduce a flexible framework for transfer learning in the context of binary classification, allowing for covariate-dependent relationships between the source and target distributions that are not required to preserve the Bayes decision boundary. Our main contributions are to derive the minimax optimal rates of convergence (up to poly-logarithmic factors) in this problem, and show that the optimal rate can be achieved by an algorithm that adapts to key aspects of the unknown transfer relationship, as well as the smoothness and tail parameters of our distributional classes. This optimal rate turns out to have several regimes, depending on the interplay between the relative sample sizes and the strength of the transfer relationship, and our algorithm achieves optimality by careful, decision tree-based calibration of local nearest-neighbour procedures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/12/2020

On the Value of Target Data in Transfer Learning

We aim to understand the value of additional labeled or unlabeled target...
research
06/07/2019

Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier

Human learners have the natural ability to use knowledge gained in one s...
research
04/11/2022

Local convergence rates of the least squares estimator with applications to transfer learning

Convergence properties of empirical risk minimizers can be conveniently ...
research
11/22/2022

Transfer Learning for Contextual Multi-armed Bandits

Motivated by a range of applications, we study in this paper the problem...
research
01/18/2021

Transferring model structure in Bayesian transfer learning for Gaussian process regression

Bayesian transfer learning (BTL) is defined in this paper as the task of...
research
03/05/2018

Marginal Singularity, and the Benefits of Labels in Covariate-Shift

We present new minimax results that concisely capture the relative benef...
research
09/22/2021

Fully probabilistic design for knowledge fusion between Bayesian filters under uniform disturbances

This paper considers the problem of Bayesian transfer learning-based kno...

Please sign up or login with your details

Forgot password? Click here to reset