Transfer Learning via ℓ_1 Regularization

by   Masaaki Takada, et al.

Machine learning algorithms typically require abundant data under a stationary environment. However, environments are nonstationary in many real-world applications. Critical issues lie in how to effectively adapt models under an ever-changing environment. We propose a method for transferring knowledge from a source domain to a target domain via ℓ_1 regularization. We incorporate ℓ_1 regularization of differences between source parameters and target parameters, in addition to an ordinary ℓ_1 regularization. Hence, our method yields sparsity for both the estimates themselves and changes of the estimates. The proposed method has a tight estimation error bound under a stationary environment, and the estimate remains unchanged from the source estimate under small residuals. Moreover, the estimate is consistent with the underlying function, even when the source estimate is mistaken due to nonstationarity. Empirical results demonstrate that the proposed method effectively balances stability and plasticity.



There are no comments yet.


page 1

page 2

page 3

page 4


Constrained Deep Transfer Feature Learning and its Applications

Feature learning with deep models has achieved impressive results for bo...

Beyond Target Networks: Improving Deep Q-learning with Functional Regularization

Target networks are at the core of recent success in Reinforcement Learn...

Unsupervised Domain Adaptation Through Transferring both the Source-Knowledge and Target-Relatedness Simultaneously

Unsupervised domain adaptation (UDA) is an emerging research topic in th...

Parameter Transfer Extreme Learning Machine based on Projective Model

Recent years, transfer learning has attracted much attention in the comm...

Transfer Regression via Pairwise Similarity Regularization

Transfer learning methods address the situation where little labeled tra...

TECM: Transfer Evidential C-means Clustering

Clustering is widely used in text analysis, natural language processing,...

Transfer Learning for Endoscopic Image Classification

In this paper we propose a method for transfer learning of endoscopic im...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.