Robust Decision Trees Against Adversarial Examples

02/27/2019
by   Hongge Chen, et al.
0

Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also vulnerable to adversarial examples and develop a novel algorithm to learn robust trees. At its core, our method aims to optimize the performance under the worst-case perturbation of input features, which leads to a max-min saddle point problem. Incorporating this saddle point objective into the decision tree building procedure is non-trivial due to the discrete nature of trees --- a naive approach to finding the best split according to this saddle point objective will take exponential time. To make our approach practical and scalable, we propose efficient tree building algorithms by approximating the inner minimizer in this saddle point problem, and present efficient implementations for classical information gain based trees as well as state-of-the-art tree boosting models such as XGBoost. Experimental results on real world datasets demonstrate that the proposed algorithms can substantially improve the robustness of tree-based models against adversarial examples.

READ FULL TEXT

page 5

page 11

page 19

research
09/08/2021

Robust Optimal Classification Trees Against Adversarial Examples

Decision trees are a popular choice of explainable model, but just like ...
research
11/23/2022

Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation

Researchers have proposed many methods for fair and robust machine learn...
research
06/10/2019

Robustness Verification of Tree-based Models

We study the robustness verification problem for tree-based models, incl...
research
06/08/2019

Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks

The problem of adversarial samples has been studied extensively for neur...
research
07/04/2022

Discrete Tree Flows via Tree-Structured Permutations

While normalizing flows for continuous data have been extensively resear...
research
06/27/2022

Adversarial Example Detection in Deployed Tree Ensembles

Tree ensembles are powerful models that are widely used. However, they a...
research
07/18/2022

Why do tree-based models still outperform deep learning on tabular data?

While deep learning has enabled tremendous progress on text and image da...

Please sign up or login with your details

Forgot password? Click here to reset