Transfer Learning via ℓ_1 Regularization

06/26/2020
by   Masaaki Takada, et al.
0

Machine learning algorithms typically require abundant data under a stationary environment. However, environments are nonstationary in many real-world applications. Critical issues lie in how to effectively adapt models under an ever-changing environment. We propose a method for transferring knowledge from a source domain to a target domain via ℓ_1 regularization. We incorporate ℓ_1 regularization of differences between source parameters and target parameters, in addition to an ordinary ℓ_1 regularization. Hence, our method yields sparsity for both the estimates themselves and changes of the estimates. The proposed method has a tight estimation error bound under a stationary environment, and the estimate remains unchanged from the source estimate under small residuals. Moreover, the estimate is consistent with the underlying function, even when the source estimate is mistaken due to nonstationarity. Empirical results demonstrate that the proposed method effectively balances stability and plasticity.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/23/2017

Constrained Deep Transfer Feature Learning and its Applications

Feature learning with deep models has achieved impressive results for bo...
06/04/2021

Beyond Target Networks: Improving Deep Q-learning with Functional Regularization

Target networks are at the core of recent success in Reinforcement Learn...
03/18/2020

Unsupervised Domain Adaptation Through Transferring both the Source-Knowledge and Target-Relatedness Simultaneously

Unsupervised domain adaptation (UDA) is an emerging research topic in th...
09/04/2018

Parameter Transfer Extreme Learning Machine based on Projective Model

Recent years, transfer learning has attracted much attention in the comm...
12/23/2017

Transfer Regression via Pairwise Similarity Regularization

Transfer learning methods address the situation where little labeled tra...
12/19/2021

TECM: Transfer Evidential C-means Clustering

Clustering is widely used in text analysis, natural language processing,...
08/24/2016

Transfer Learning for Endoscopic Image Classification

In this paper we propose a method for transfer learning of endoscopic im...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.