Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions

02/01/2021
by   Quentin Renau, et al.
0

Facilitated by the recent advances of Machine Learning (ML), the automated design of optimization heuristics is currently shaking up evolutionary computation (EC). Where the design of hand-picked guidelines for choosing a most suitable heuristic has long dominated research activities in the field, automatically trained heuristics are now seen to outperform human-derived choices even for well-researched optimization tasks. ML-based EC is therefore not any more a futuristic vision, but has become an integral part of our community. A key criticism that ML-based heuristics are often faced with is their potential lack of explainability, which may hinder future developments. This applies in particular to supervised learning techniques which extrapolate algorithms' performance based on exploratory landscape analysis (ELA). In such applications, it is not uncommon to use dozens of problem features to build the models underlying the specific algorithm selection or configuration task. Our goal in this work is to analyze whether this many features are indeed needed. Using the classification of the BBOB test functions as testbed, we show that a surprisingly small number of features – often less than four – can suffice to achieve a 98% accuracy. Interestingly, the number of features required to meet this threshold is found to decrease with the problem dimension. We show that the classification accuracy transfers to settings in which several instances are involved in training and testing. In the leave-one-instance-out setting, however, classification accuracy drops significantly, and the transformation-invariance of the features becomes a decisive success factor.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/22/2022

Explainable Landscape Analysis in Automated Algorithm Performance Prediction

Predicting the performance of an optimization algorithm on a new problem...
06/19/2020

Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy

Exploratory landscape analysis (ELA) supports supervised learning approa...
10/08/2017

Structural Feature Selection for Event Logs

We consider the problem of classifying business process instances based ...
10/22/2021

Explainable Landscape-Aware Optimization Performance Prediction

Efficient solving of an unseen optimization problem is related to approp...
06/27/2022

Toward an ImageNet Library of Functions for Global Optimization Benchmarking

Knowledge of search-landscape features of BlackBox Optimization (BBO) pr...
09/30/2020

Linear Matrix Factorization Embeddings for Single-objective Optimization Landscapes

Automated per-instance algorithm selection and configuration have shown ...
02/10/2021

Towards Feature-Based Performance Regression Using Trajectory Data

Black-box optimization is a very active area of research, with many new ...