Locally Constant Networks

09/30/2019
by   Guang-He Lee, et al.
0

We show how neural models can be used to realize piece-wise constant functions such as decision trees. Our approach builds on ReLU networks that are piece-wise linear and hence their associated gradients with respect to the inputs are locally constant. We formally establish the equivalence between the classes of locally constant networks and decision trees. Moreover, we highlight several advantageous properties of locally constant networks, including how they realize decision trees with parameter sharing across branching / leaves. Indeed, only M neurons suffice to implicitly model an oblique decision tree with 2^M leaf nodes. The neural representation also enables us to adopt many tools developed for deep networks (e.g., DropConnect (Wan et al. 2013)) while implicitly training decision trees. We demonstrate that our method outperforms alternative techniques for training oblique decision trees in the context of molecular property classification and regression tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/11/2022

Neural Networks are Decision Trees

In this manuscript, we show that any neural network having piece-wise li...
research
04/28/2021

Deep Neural Network as an alternative to Boosted Decision Trees for PID

In this paper we recreate, and improve, the binary classification method...
research
06/04/2014

Local Decorrelation For Improved Detection

Even with the advent of more sophisticated, data-hungry methods, boosted...
research
01/19/2023

Towards Rigorous Understanding of Neural Networks via Semantics-preserving Transformations

In this paper we present an algebraic approach to the precise and global...
research
03/06/2021

visTree: Visualization of Subgroups for a Decision Tree

Decision trees are flexible prediction models which are constructed to q...
research
01/14/2019

Towards Using Context-Dependent Symbols in CTC Without State-Tying Decision Trees

Deep neural acoustic models benefit from context dependent modeling of o...
research
02/06/2023

Diagnosis of Constant Faults in Switching Networks

In this paper, we study decision trees for diagnosis of constant faults ...

Please sign up or login with your details

Forgot password? Click here to reset