Parametric Programming Approach for Powerful Lasso Selective Inference without Conditioning on Signs
share
In the past few years, Selective Inference (SI) has been actively studied for inference on the features of linear models that are adaptively selected by feature selection methods. A seminal work is proposed by Lee et al. (2016) in the case of the Lasso. The basic idea of SI is to make inference conditional on the selection event. In Lee et al. (2016), the authors proposed a tractable way to conduct inference conditional on the selected features and their signs. Unfortunately, additionally conditioning on the signs leads to low statistical power because of over-conditioning. To improve the power, a current available possible solution is to remove the conditioning on signs by considering the union of an exponentially large number of all possible sign vectors, which leads to an unrealistically large amount of computational cost unless the number of selected features is sufficiently small. To address this problem, we propose an efficient method to characterize the selection event without conditioning on signs by using parametric programming. The main idea is to compute the continuum path of Lasso solutions in the direction of a test statistic, and identify the subset of data space corresponding to the feature selection event by following the solution path. We conduct several experiments to demonstrate the effectiveness and efficiency of our proposed method.
READ FULL TEXT