Log In Sign Up

Flexible, Non-parametric Modeling Using Regularized Neural Networks

by   Oskar Allerbo, et al.

Non-parametric regression, such as generalized additive models (GAMs), is able to capture complex data dependencies in a flexible, yet interpretable way. However, choosing the format of the additive components often requires non-trivial data exploration. Here, we propose an alternative to GAMs, PrAda-net, which uses a one hidden layer neural network, trained with proximal gradient descent and adaptive lasso. PrAda-net automatically adjusts the size and architecture of the neural network to capture the complexity and structure of the underlying data generative model. The compact network obtained by PrAda-net can be translated to additive model components, making it suitable for non-parametric statistical modelling with automatic model selection. We demonstrate PrAda-net on simulated data, where we compare the test error performance, variable importance and variable subset identification properties of PrAda-net to other lasso-based approaches. We also apply Prada-net to the massive U.K. black smoke data set, to demonstrate the capability of using Prada-net as an alternative to GAMs. In contrast to GAMs, which often require domain knowledge to select the functional forms of the additive components, Prada-net requires no such pre-selection while still resulting in interpretable additive components.


page 1

page 2

page 3

page 4


Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity

Interpretable machine learning has demonstrated impressive performance w...

Generalized Additive Model Selection

We introduce GAMSEL (Generalized Additive Model Selection), a penalized ...

Sparse additive Gaussian process with soft interactions

Additive nonparametric regression models provide an attractive tool for ...

PSF–NET: A Non-parametric Point Spread Function Model for Ground Based Optical Telescopes

Ground based optical telescopes are seriously affected by atmospheric tu...

Non-parametric Bayesian Learning with Deep Learning Structure and Its Applications in Wireless Networks

In this paper, we present an infinite hierarchical non-parametric Bayesi...

Selective Inference for L_2-Boosting

We review several recently proposed post-selection inference frameworks ...

Church: a language for generative models

We introduce Church, a universal language for describing stochastic gene...