Bayesian group latent factor analysis with structured sparsity

11/11/2014
by   Shiwen Zhao, et al.
0

Latent factor models are the canonical statistical tool for exploratory analyses of low-dimensional linear structure for an observation matrix with p features across n samples. We develop a structured Bayesian group factor analysis model that extends the factor model to multiple coupled observation matrices; in the case of two observations, this reduces to a Bayesian model of canonical correlation analysis. The main contribution of this work is to carefully define a structured Bayesian prior that encourages both element-wise and column-wise shrinkage and leads to desirable behavior on high-dimensional data. In particular, our model puts a structured prior on the joint factor loading matrix, regularizing at three levels, which enables element-wise sparsity and unsupervised recovery of latent factors corresponding to structured variance across arbitrary subsets of the observations. In addition, our structured prior allows for both dense and sparse latent factors so that covariation among either all features or only a subset of features can both be recovered. We use fast parameter-expanded expectation-maximization for parameter estimation in this model. We validate our method on both simulated data with substantial structure and real data, comparing against a number of state-of-the-art approaches. These results illustrate useful properties of our model, including i) recovering sparse signal in the presence of dense effects; ii) the ability to scale naturally to large numbers of observations; iii) flexible observation- and factor-specific regularization to recover factors with a wide variety of sparsity levels and percentage of variance explained; and iv) tractable inference that scales to modern genomic and document data sizes.

READ FULL TEXT
research
11/21/2014

Group Factor Analysis

Factor analysis provides linear factors that describe relationships betw...
research
03/14/2021

A two-way factor model for high-dimensional matrix data

In this article, we introduce a two-way factor model for a high-dimensio...
research
06/22/2020

Latent feature sharing: an adaptive approach to linear decomposition models

Latent feature models are canonical tools for exploratory analysis in cl...
research
03/17/2020

Nonparametric Deconvolution Models

We describe nonparametric deconvolution models (NDMs), a family of Bayes...
research
04/13/2017

Infinite Sparse Structured Factor Analysis

Matrix factorisation methods decompose multivariate observations as line...
research
03/10/2015

Learning the Structure for Structured Sparsity

Structured sparsity has recently emerged in statistics, machine learning...
research
11/11/2019

Bayesian Non-Parametric Factor Analysis for Longitudinal Spatial Surfaces

We introduce a Bayesian non-parametric spatial factor analysis model wit...

Please sign up or login with your details

Forgot password? Click here to reset