Sparse linear regression – CLuP achieves the ideal exact ML

by   Mihailo Stojnic, et al.

In this paper we revisit one of the classical statistical problems, the so-called sparse maximum-likelihood (ML) linear regression. As a way of attacking this type of regression, we present a novel CLuP mechanism that to a degree relies on the Random Duality Theory (RDT) based algorithmic machinery that we recently introduced in <cit.>. After the initial success that the CLuP exhibited in achieving the exact ML performance while maintaining excellent computational complexity related properties in MIMO ML detection in <cit.>, one would naturally expect that a similar type of success can be achieved in other ML considerations. The results that we present here confirm that such an expectation is indeed reasonable. In particular, within the sparse regression context, the introduced CLuP mechanism indeed turns out to be able to achieve the ideal ML performance. Moreover, it can substantially outperform some of the most prominent earlier state of the art algorithmic concepts, among them even the variants of the famous LASSO and SOCP from <cit.>. Also, our recent results presented in <cit.> showed that the CLuP has excellent large-scale and the so-called rephasing abilities. Since such large-scale algorithmic features are possibly even more desirable within the sparse regression context we here also demonstrate that the basic CLuP ideas can be reformulated to enable solving with a relative ease the regression problems with several thousands of unknowns.



page 1

page 2

page 3

page 4


Rephased CLuP

In <cit.> we introduced CLuP, a Random Duality Theory (RDT) based algori...

Complexity analysis of the Controlled Loosening-up (CLuP) algorithm

In our companion paper <cit.> we introduced a powerful mechanism that we...

Algorithmic random duality theory – large scale CLuP

Based on our Random Duality Theory (RDT), in a sequence of our recent pa...

Starting CLuP with polytope relaxation

The Controlled Loosening-up (CLuP) mechanism that we recently introduced...

Controlled Loosening-up (CLuP) – achieving exact MIMO ML in polynomial time

In this paper we attack one of the most fundamental signal processing/in...

Distribution free testing for linear regression. Extension to general parametric regression

Recently a distribution free approach for testing parametric hypotheses ...

An Intrinsic Treatment of Stochastic Linear Regression

Linear regression is perhaps one of the most popular statistical concept...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.