Log In Sign Up

Constrained Bayesian Inference for Low Rank Multitask Learning

by   Oluwasanmi Koyejo, et al.

We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply the proposed constrained Bayesian inference approach to multitask learning subject to rank constraints on the weight matrix. Further, constrained parameter estimation is applied to recover the sparse conditional independence structure encoded by prior precision matrices. Our approach is motivated by reverse inference for high dimensional functional neuroimaging, a domain where the high dimensionality and small number of examples requires the use of constraints to ensure meaningful and effective models. For this application, we propose a model that jointly learns a weight matrix and the prior inverse covariance structure between different tasks. We present experimental validation showing that the proposed approach outperforms strong baseline models in terms of predictive performance and structure recovery.


page 1

page 2

page 3

page 4


Bayesian Inference with Projected Densities

Constraints are a natural choice for prior information in Bayesian infer...

The trace norm constrained matrix-variate Gaussian process for multitask bipartite ranking

We propose a novel hierarchical model for multitask bipartite ranking. T...

General Bayesian Inference over the Stiefel Manifold via the Givens Transform

We introduce the Givens Transform, a novel transform between the space o...

Overlap matrix concentration in optimal Bayesian inference

We consider models of Bayesian inference of signals with vectorial compo...

A Generative Deep Recurrent Model for Exchangeable Data

We present a novel model architecture which leverages deep learning tool...

A conditional one-output likelihood formulation for multitask Gaussian processes

Multitask Gaussian processes (MTGP) are the Gaussian process (GP) framew...

Bayesian Eigenvalue Regularization via Cumulative Shrinkage Process

This study proposes a novel hierarchical prior for inferring possibly lo...