Scalable Algorithms for Learning High-Dimensional Linear Mixed Models

03/12/2018
by   Zilong Tan, et al.
0

Linear mixed models (LMMs) are used extensively to model dependecies of observations in linear regression and are used extensively in many application areas. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational complexity which depends at least linearly on the dimension p of the covariates, and often use heuristics that do not offer theoretical guarantees. We present scalable algorithms for learning high-dimensional LMMs with sublinear computational complexity dependence on p. Key to our approach are novel dual estimators which use only kernel functions of the data, and fast computational techniques based on the subsampled randomized Hadamard transform. We provide theoretical guarantees for our learning algorithms, demonstrating the robustness of parameter estimation. Finally, we complement the theory with experiments on large synthetic and real data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/10/2013

Feature Selection with Annealing for Computer Vision and Big Data Learning

Many computer vision and medical imaging problems are faced with learnin...
research
06/20/2019

Active Linear Regression

We consider the problem of active linear regression where a decision mak...
research
04/13/2015

Adaptive Randomized Dimension Reduction on Massive Data

The scalability of statistical estimators is of increasing importance in...
research
07/11/2013

Fast gradient descent for drifting least squares regression, with application to bandits

Online learning algorithms require to often recompute least squares regr...
research
06/02/2022

Sparse Mixed Linear Regression with Guarantees: Taming an Intractable Problem with Invex Relaxation

In this paper, we study the problem of sparse mixed linear regression on...
research
11/17/2022

Validation Diagnostics for SBI algorithms based on Normalizing Flows

Building on the recent trend of new deep generative models known as Norm...
research
02/25/2017

Efficient Learning of Mixed Membership Models

We present an efficient algorithm for learning mixed membership models w...

Please sign up or login with your details

Forgot password? Click here to reset