GAGA: Deciphering Age-path of Generalized Self-paced Regularizer

09/15/2022
by   Xingyu Qu, et al.
3

Nowadays self-paced learning (SPL) is an important machine learning paradigm that mimics the cognitive process of humans and animals. The SPL regime involves a self-paced regularizer and a gradually increasing age parameter, which plays a key role in SPL but where to optimally terminate this process is still non-trivial to determine. A natural idea is to compute the solution path w.r.t. age parameter (i.e., age-path). However, current age-path algorithms are either limited to the simplest regularizer, or lack solid theoretical understanding as well as computational efficiency. To address this challenge, we propose a novel Generalized Age-path Algorithm (GAGA) for SPL with various self-paced regularizers based on ordinary differential equations (ODEs) and sets control, which can learn the entire solution spectrum w.r.t. a range of age parameters. To the best of our knowledge, GAGA is the first exact path-following algorithm tackling the age-path for general self-paced regularizer. Finally the algorithmic steps of classic SVM and Lasso are described in detail. We demonstrate the performance of GAGA on real-world datasets, and find considerable speedup between our algorithm and competing baselines.

READ FULL TEXT

page 25

page 26

page 27

page 28

page 29

page 30

page 31

research
06/01/2016

Self-Paced Learning: an Implicit Regularization Perspective

Self-paced learning (SPL) mimics the cognitive mechanism of humans and a...
research
03/05/2020

Linear time dynamic programming for the exact path of optimal models selected from a finite set

Many learning algorithms are formulated in terms of finding model parame...
research
10/18/2022

SVLDL: Improved Speaker Age Estimation Using Selective Variance Label Distribution Learning

Estimating age from a single speech is a classic and challenging topic. ...
research
10/07/2017

Deep Self-Paced Learning for Person Re-Identification

Person re-identification (Re-ID) usually suffers from noisy samples with...
research
06/11/2022

An Algorithm for Exact Numerical Age-of-Information Evaluation in Multi-Agent Systems

We present an algorithm for the numerical evaluation of the state-space ...
research
03/27/2009

A Combinatorial Algorithm to Compute Regularization Paths

For a wide variety of regularization methods, algorithms computing the e...

Please sign up or login with your details

Forgot password? Click here to reset