Supervised Dictionary Learning with Auxiliary Covariates

06/14/2022
by   Joowon Lee, et al.
0

Supervised dictionary learning (SDL) is a classical machine learning method that simultaneously seeks feature extraction and classification tasks, which are not necessarily a priori aligned objectives. The goal of SDL is to learn a class-discriminative dictionary, which is a set of latent feature vectors that can well-explain both the features as well as labels of observed data. In this paper, we provide a systematic study of SDL, including the theory, algorithm, and applications of SDL. First, we provide a novel framework that `lifts' SDL as a convex problem in a combined factor space and propose a low-rank projected gradient descent algorithm that converges exponentially to the global minimizer of the objective. We also formulate generative models of SDL and provide global estimation guarantees of the true parameters depending on the hyperparameter regime. Second, viewed as a nonconvex constrained optimization problem, we provided an efficient block coordinate descent algorithm for SDL that is guaranteed to find an ε-stationary point of the objective in O(ε^-1(logε^-1)^2) iterations. For the corresponding generative model, we establish a novel non-asymptotic local consistency result for constrained and regularized maximum likelihood estimation problems, which may be of independent interest. Third, we apply SDL for imbalanced document classification by supervised topic modeling and also for pneumonia detection from chest X-ray images. We also provide simulation studies to demonstrate that SDL becomes more effective when there is a discrepancy between the best reconstructive and the best discriminative dictionaries.

READ FULL TEXT

page 26

page 27

page 28

page 30

page 32

page 33

research
05/03/2016

Decentralized Dynamic Discriminative Dictionary Learning

We consider discriminative dictionary learning in a distributed online s...
research
06/04/2023

Complexity of Block Coordinate Descent with Proximal Regularization and Applications to Wasserstein CP-dictionary Learning

We consider the block coordinate descent methods of Gauss-Seidel type wi...
research
02/05/2018

Weakly-supervised Dictionary Learning

We present a probabilistic modeling and inference framework for discrimi...
research
05/24/2023

Personalized Dictionary Learning for Heterogeneous Datasets

We introduce a relevant yet challenging problem named Personalized Dicti...
research
08/17/2012

Information-theoretic Dictionary Learning for Image Classification

We present a two-stage approach for learning dictionaries for object cla...
research
12/12/2018

An efficient supervised dictionary learning method for audio signal recognition

Machine hearing or listening represents an emerging area. Conventional a...
research
04/24/2018

On Learning Sparsely Used Dictionaries from Incomplete Samples

Most existing algorithms for dictionary learning assume that all entries...

Please sign up or login with your details

Forgot password? Click here to reset