Sparse Mixed Linear Regression with Guarantees: Taming an Intractable Problem with Invex Relaxation

06/02/2022
by   Adarsh Barik, et al.
0

In this paper, we study the problem of sparse mixed linear regression on an unlabeled dataset that is generated from linear measurements from two different regression parameter vectors. Since the data is unlabeled, our task is not only to figure out a good approximation of the regression parameter vectors but also to label the dataset correctly. In its original form, this problem is NP-hard. The most popular algorithms to solve this problem (such as Expectation-Maximization) have a tendency to stuck at local minima. We provide a novel invex relaxation for this intractable problem which leads to a solution with provable theoretical guarantees. This relaxation enables exact recovery of data labels. Furthermore, we recover a close approximation of the regression parameter vectors which match the true parameter vectors in support and sign. Our formulation uses a carefully constructed primal dual witnesses framework for the invex problem. Furthermore, we show that the sample complexity of our method is only logarithmic in terms of the dimension of the regression parameter vectors.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/19/2021

Fair Sparse Regression with Clustering: An Invex Relaxation for a Combinatorial Problem

In this paper, we study the problem of fair sparse regression on a biase...
research
12/21/2022

A Theoretical Study of The Effects of Adversarial Attacks on Sparse Regression

This paper analyzes ℓ_1 regularized linear regression under the challeng...
research
03/03/2023

Statistical-Computational Tradeoffs in Mixed Sparse Linear Regression

We consider the problem of mixed sparse linear regression with two compo...
research
03/21/2019

Convergence of Parameter Estimates for Regularized Mixed Linear Regression Models

We consider Mixed Linear Regression (MLR), where training data have bee...
research
04/23/2020

Alternating Minimization Converges Super-Linearly for Mixed Linear Regression

We address the problem of solving mixed random linear equations. We have...
research
06/22/2023

Outlier-robust Estimation of a Sparse Linear Model Using Invexity

In this paper, we study problem of estimating a sparse regression vector...
research
03/12/2018

Scalable Algorithms for Learning High-Dimensional Linear Mixed Models

Linear mixed models (LMMs) are used extensively to model dependecies of ...

Please sign up or login with your details

Forgot password? Click here to reset