DeepAI AI Chat
Log In Sign Up

Towards Reproducible Empirical Research in Meta-Learning

08/30/2018
by   Adriano Rivolli, et al.
Universidade de São Paulo
Universidade do Porto
UTFPR
TU Eindhoven
0

Meta-learning is increasingly used to support the recommendation of machine learning algorithms and their configurations. Such recommendations are made based on meta-data, consisting of performance evaluations of algorithms on prior datasets, as well as characterizations of these datasets. These characterizations, also called meta-features, describe properties of the data which are predictive for the performance of machine learning algorithms trained on them. Unfortunately, despite being used in a large number of studies, meta-features are not uniformly described and computed, making many empirical studies irreproducible and hard to compare. This paper aims to remedy this by systematizing and standardizing data characterization measures used in meta-learning, and performing an in-depth analysis of their utility. Moreover, it presents MFE, a new tool for extracting meta-features from datasets and identify more subtle reproducibility issues in the literature, proposing guidelines for data characterization that strengthen reproducible empirical research in meta-learning.

READ FULL TEXT
08/27/2020

learn2learn: A Library for Meta-Learning Research

Meta-learning researchers face two fundamental issues in their empirical...
06/28/2021

Explaining the Performance of Multi-label Classification Methods with Data Set Properties

Meta learning generalizes the empirical experience with different learni...
01/19/2021

Learning Abstract Task Representations

A proper form of data characterization can guide the process of learning...
06/07/2021

Evaluating Meta-Feature Selection for the Algorithm Recommendation Problem

With the popularity of Machine Learning (ML) solutions, algorithms and d...
02/22/2022

Enabling Reproducibility and Meta-learning Through a Lifelong Database of Experiments (LDE)

Artificial Intelligence (AI) development is inherently iterative and exp...
08/26/2019

An Introduction to Advanced Machine Learning : Meta Learning Algorithms, Applications and Promises

In [1, 2], we have explored the theoretical aspects of feature extractio...
03/08/2023

Meta-learning Control Variates: Variance Reduction with Limited Data

Control variates can be a powerful tool to reduce the variance of Monte ...