Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation

11/23/2022
by   Josh Gardner, et al.
0

Researchers have proposed many methods for fair and robust machine learning, but comprehensive empirical evaluation of their subgroup robustness is lacking. In this work, we address this gap in the context of tabular data, where sensitive subgroups are clearly-defined, real-world fairness problems abound, and prior works often do not compare to state-of-the-art tree-based models as baselines. We conduct an empirical comparison of several previously-proposed methods for fair and robust learning alongside state-of-the-art tree-based methods and other baselines. Via experiments with more than 340,000 model configurations on eight datasets, we show that tree-based methods have strong subgroup robustness, even when compared to robustness- and fairness-enhancing methods. Moreover, the best tree-based models tend to show good performance over a range of metrics, while robust or group-fair models can show brittleness, with significant performance differences across different metrics for a fixed model. We also demonstrate that tree-based models show less sensitivity to hyperparameter configurations, and are less costly to train. Our work suggests that tree-based ensemble models make an effective baseline for tabular data, and are a sensible default when subgroup robustness is desired. For associated code and detailed results, see https://github.com/jpgard/subgroup-robustness-grows-on-trees .

READ FULL TEXT

page 2

page 36

page 37

research
02/27/2019

Robust Decision Trees Against Adversarial Examples

Although adversarial examples and model robustness have been extensively...
research
07/18/2022

Why do tree-based models still outperform deep learning on tabular data?

While deep learning has enabled tremendous progress on text and image da...
research
06/15/2023

Harvard Glaucoma Fairness: A Retinal Nerve Disease Dataset for Fairness Learning and Fair Identity Normalization

Fairness in machine learning is important for societal well-being, but l...
research
12/21/2017

Fair Forests: Regularized Tree Induction to Minimize Model Bias

The potential lack of fairness in the outputs of machine learning algori...
research
11/25/2018

Intersectionality: Multiple Group Fairness in Expectation Constraints

Group fairness is an important concern for machine learning researchers,...
research
10/30/2020

A Critical Assessment of State-of-the-Art in Entity Alignment

In this work, we perform an extensive investigation of two state-of-the-...
research
04/29/2022

A study of tree-based methods and their combination

Tree-based methods are popular machine learning techniques used in vario...

Please sign up or login with your details

Forgot password? Click here to reset