
Transfer Learning Can Outperform the True Prior in Double Descent Regularization
We study a fundamental transfer learning process from source to target l...
read it

Phase Transitions in Transfer Learning for HighDimensional Perceptrons
Transfer learning seeks to improve the generalization performance of a t...
read it

EmojiBased Transfer Learning for Sentiment Tasks
Sentiment tasks such as hate speech detection and sentiment analysis, es...
read it

Subspace Fitting Meets Regression: The Effects of Supervision and Orthonormality Constraints on Double Descent of Generalization Errors
We study the linear subspace fitting problem in the overparameterized se...
read it

Transfer Learning for Linear Regression: a Statistical Test of Gain
Transfer learning, also referred as knowledge transfer, aims at reusing ...
read it

Transferring model structure in Bayesian transfer learning for Gaussian process regression
Bayesian transfer learning (BTL) is defined in this paper as the task of...
read it

Generalization Error of Generalized Linear Models in High Dimensions
At the heart of machine learning lies the question of generalizability o...
read it
Double Double Descent: On Generalization Errors in Transfer Learning between Linear Regression Tasks
We study the transfer learning process between two linear regression problems. An important and timely special case is when the regressors are overparameterized and perfectly interpolate their training data. We examine a parameter transfer mechanism whereby a subset of the parameters of the target task solution are constrained to the values learned for a related source task. We analytically characterize the generalization error of the target task in terms of the salient factors in the transfer learning architecture, i.e., the number of examples available, the number of (free) parameters in each of the tasks, the number of parameters transferred from the source to target task, and the correlation between the two tasks. Our nonasymptotic analysis shows that the generalization error of the target task follows a twodimensional double descent trend (with respect to the number of free parameters in each of the tasks) that is controlled by the transfer learning factors. Our analysis points to specific cases where the transfer of parameters is beneficial.
READ FULL TEXT
Comments
There are no comments yet.