Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees

04/15/2022
by   Wenying Deng, et al.
0

We develop a simple and unified framework for nonlinear variable selection that incorporates model uncertainty and is compatible with a wide range of machine learning models (e.g., tree ensembles, kernel methods and neural network). In particular, for a learned nonlinear model f(𝐱), we consider quantifying the importance of an input variable 𝐱^j using the integrated gradient measure ψ_j = ‖∂/∂𝐱^j f(𝐱)‖^2_2. We then (1) provide a principled approach for quantifying variable selection uncertainty by deriving its posterior distribution, and (2) show that the approach is generalizable even to non-differentiable models such as tree ensembles. Rigorous Bayesian nonparametric theorems are derived to guarantee the posterior consistency and asymptotic uncertainty of the proposed approach. Extensive simulation confirms that the proposed algorithm outperforms existing classic and recent variable selection methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/25/2021

Consistent Sparse Deep Learning: Theory and Computation

Deep learning has been the engine powering many successes of data scienc...
research
12/06/2007

Kernels and Ensembles: Perspectives on Statistical Learning

Since their emergence in the 1990's, the support vector machine and the ...
research
12/10/2018

Variational Nonparametric Discriminant Analysis

Variable selection and classification methods are common objectives in t...
research
08/04/2019

Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting

When randomized ensemble methods such as bagging and random forests are ...
research
11/05/2020

Nonparametric Variable Screening with Optimal Decision Stumps

Decision trees and their ensembles are endowed with a rich set of diagno...

Please sign up or login with your details

Forgot password? Click here to reset