Sparse linear regression – CLuP achieves the ideal exact ML

11/23/2020
by   Mihailo Stojnic, et al.
0

In this paper we revisit one of the classical statistical problems, the so-called sparse maximum-likelihood (ML) linear regression. As a way of attacking this type of regression, we present a novel CLuP mechanism that to a degree relies on the Random Duality Theory (RDT) based algorithmic machinery that we recently introduced in <cit.>. After the initial success that the CLuP exhibited in achieving the exact ML performance while maintaining excellent computational complexity related properties in MIMO ML detection in <cit.>, one would naturally expect that a similar type of success can be achieved in other ML considerations. The results that we present here confirm that such an expectation is indeed reasonable. In particular, within the sparse regression context, the introduced CLuP mechanism indeed turns out to be able to achieve the ideal ML performance. Moreover, it can substantially outperform some of the most prominent earlier state of the art algorithmic concepts, among them even the variants of the famous LASSO and SOCP from <cit.>. Also, our recent results presented in <cit.> showed that the CLuP has excellent large-scale and the so-called rephasing abilities. Since such large-scale algorithmic features are possibly even more desirable within the sparse regression context we here also demonstrate that the basic CLuP ideas can be reformulated to enable solving with a relative ease the regression problems with several thousands of unknowns.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

11/23/2020

Rephased CLuP

In <cit.> we introduced CLuP, a Random Duality Theory (RDT) based algori...
09/03/2019

Complexity analysis of the Controlled Loosening-up (CLuP) algorithm

In our companion paper <cit.> we introduced a powerful mechanism that we...
11/23/2020

Algorithmic random duality theory – large scale CLuP

Based on our Random Duality Theory (RDT), in a sequence of our recent pa...
09/03/2019

Starting CLuP with polytope relaxation

The Controlled Loosening-up (CLuP) mechanism that we recently introduced...
09/03/2019

Controlled Loosening-up (CLuP) – achieving exact MIMO ML in polynomial time

In this paper we attack one of the most fundamental signal processing/in...
02/07/2020

Distribution free testing for linear regression. Extension to general parametric regression

Recently a distribution free approach for testing parametric hypotheses ...
09/16/2020

An Intrinsic Treatment of Stochastic Linear Regression

Linear regression is perhaps one of the most popular statistical concept...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.