DeepAI AI Chat
Log In Sign Up

Interpretable Selection and Visualization of Features and Interactions Using Bayesian Forests

by   Viktoriya Krakovna, et al.

It is becoming increasingly important for machine learning methods to make predictions that are interpretable as well as accurate. In many practical applications, it is of interest which features and feature interactions are relevant to the prediction task. We present a novel method, Selective Bayesian Forest Classifier, that strikes a balance between predictive power and interpretability by simultaneously performing classification, feature selection, feature interaction detection and visualization. It builds parsimonious yet flexible models using tree-structured Bayesian networks, and samples an ensemble of such models using Markov chain Monte Carlo. We build in feature selection by dividing the trees into two groups according to their relevance to the outcome of interest. Our method performs competitively on classification and feature selection benchmarks in low and high dimensions, and includes a visualization tool that provides insight into relevant features and interactions.


page 5

page 6

page 7

page 8


Interpretability with full complexity by constraining feature information

Interpretability is a pressing issue for machine learning. Common approa...

Comparing interpretability and explainability for feature selection

A common approach for feature selection is to examine the variable impor...

Feature Interactions in XGBoost

In this paper, we investigate how feature interactions can be identified...

Random Subspace with Trees for Feature Selection Under Memory Constraints

Dealing with datasets of very high dimension is a major challenge in mac...

A Bayesian machine scientist to aid in the solution of challenging scientific problems

Closed-form, interpretable mathematical models have been instrumental fo...

A User-Guided Bayesian Framework for Ensemble Feature Selection in Life Science Applications (UBayFS)

Training machine learning models on high-dimensional datasets is a chall...

Code Repositories


Selective Bayesian Forest Classifier - R package for simultaneous feature selection and classification. See paper:

view repo