Consistent Parameter Estimation for LASSO and Approximate Message Passing
share
We consider the problem of recovering a vector β_o ∈R^p from n random and noisy linear observations y= Xβ_o + w, where X is the measurement matrix and w is noise. The LASSO estimate is given by the solution to the optimization problem β̂_λ = _β1/2y-Xβ_2^2 + λβ_1. Among the iterative algorithms that have been proposed for solving this optimization problem, approximate message passing (AMP) has attracted attention for its fast convergence. Despite significant progress in the theoretical analysis of the estimates of LASSO and AMP, little is known about their behavior as a function of the regularization parameter λ, or the thereshold parameters τ^t. For instance the following basic questions have not yet been studied in the literature: (i) How does the size of the active set β̂^λ_0/p behave as a function of λ? (ii) How does the mean square error β̂_λ - β_o_2^2/p behave as a function of λ? (iii) How does β^t - β_o _2^2/p behave as a function of τ^1, ..., τ^t-1? Answering these questions will help in addressing practical challenges regarding the optimal tuning of λ or τ^1, τ^2, .... This paper answers these questions in the asymptotic setting and shows how these results can be employed in deriving simple and theoretically optimal approaches for tuning the parameters τ^1, ..., τ^t for AMP or λ for LASSO. It also explores the connection between the optimal tuning of the parameters of AMP and the optimal tuning of LASSO.
READ FULL TEXT
