An Efficient Adversarial Attack for Tree Ensembles

by   Chong Zhang, et al.

We study the problem of efficient adversarial attacks on tree based ensembles such as gradient boosting decision trees (GBDTs) and random forests (RFs). Since these models are non-continuous step functions and gradient does not exist, most existing efficient adversarial attacks are not applicable. Although decision-based black-box attacks can be applied, they cannot utilize the special structure of trees. In our work, we transform the attack problem into a discrete search problem specially designed for tree ensembles, where the goal is to find a valid "leaf tuple" that leads to mis-classification while having the shortest distance to the original input. With this formulation, we show that a simple yet effective greedy algorithm can be applied to iteratively optimize the adversarial example by moving the leaf tuple to its neighborhood within hamming distance 1. Experimental results on several large GBDT and RF models with up to hundreds of trees demonstrate that our method can be thousands of times faster than the previous mixed-integer linear programming (MILP) based approach, while also providing smaller (better) adversarial examples than decision-based black-box attacks on general ℓ_p (p=1, 2, ∞) norm perturbations. Our code is available at


page 1

page 2

page 3

page 4


Output Diversified Initialization for Adversarial Attacks

Adversarial examples are often constructed by iteratively refining a ran...

Robustness Verification of Tree-based Models

We study the robustness verification problem for tree-based models, incl...

Additive Tree Ensembles: Reasoning About Potential Instances

Imagine being able to ask questions to a black box model such as "Which ...

Scalable Whitebox Attacks on Tree-based Models

Adversarial robustness is one of the essential safety criteria for guara...

Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks

The problem of adversarial samples has been studied extensively for neur...

Efficient Training of Robust Decision Trees Against Adversarial Examples

In the present day we use machine learning for sensitive tasks that requ...

Learning Nonlinear Functions Using Regularized Greedy Forest

We consider the problem of learning a forest of nonlinear decision rules...