Information Projection and Approximate Inference for Structured Sparse Variables

07/12/2016
by   Rajiv Khanna, et al.
0

Approximate inference via information projection has been recently introduced as a general-purpose approach for efficient probabilistic inference given sparse variables. This manuscript goes beyond classical sparsity by proposing efficient algorithms for approximate inference via information projection that are applicable to any structure on the set of variables that admits enumeration using a matroid. We show that the resulting information projection can be reduced to combinatorial submodular optimization subject to matroid constraints. Further, leveraging recent advances in submodular optimization, we provide an efficient greedy algorithm with strong optimization-theoretic guarantees. The class of probabilistic models that can be expressed in this way is quite broad and, as we show, includes group sparse regression, group sparse principal components analysis and sparse canonical correlation analysis, among others. Moreover, empirical results on simulated data and high dimensional neuroimaging data highlight the superior performance of the information projection approach as compared to established baselines for a range of probabilistic models.

READ FULL TEXT
research
05/30/2017

Sparse canonical correlation analysis

Canonical correlation analysis was proposed by Hotelling [6] and it meas...
research
04/01/2018

Sparse Principal Component Analysis via Variable Projection

Sparse principal component analysis (SPCA) has emerged as a powerful tec...
research
01/22/2014

Collaborative Regression

We consider the scenario where one observes an outcome variable and sets...
research
09/10/2021

Latent space projection predictive inference

Given a reference model that includes all the available variables, proje...
research
09/30/2016

Technical Report: Graph-Structured Sparse Optimization for Connected Subgraph Detection

Structured sparse optimization is an important and challenging problem f...
research
12/05/2012

Sparse seismic imaging using variable projection

We consider an important class of signal processing problems where the s...
research
02/19/2016

Structured Sparse Regression via Greedy Hard-Thresholding

Several learning applications require solving high-dimensional regressio...

Please sign up or login with your details

Forgot password? Click here to reset