Meta-learning from Learning Curves Challenge: Lessons learned from the First Round and Design of the Second Round

08/04/2022

∙

Meta-learning from learning curves is an important yet often neglected research area in the Machine Learning community. We introduce a series of Reinforcement Learning-based meta-learning challenges, in which an agent searches for the best suited algorithm for a given dataset, based on feedback of learning curves from the environment. The first round attracted participants both from academia and industry. This paper analyzes the results of the first round (accepted to the competition program of WCCI 2022), to draw insights into what makes a meta-learner successful at learning from learning curves. With the lessons learned from the first round and the feedback from the participants, we have designed the second round of our challenge with a new protocol and a new meta-dataset. The second round of our challenge is accepted at the AutoML-Conf 2022 and currently ongoing .

READ FULL TEXT

Meta-learning from Learning Curves Challenge: Lessons learned from the First Round and Design of the Second Round

NeurIPS'22 Cross-Domain MetaDL competition: Design and baseline results

Learning to Reason in Round-based Games: Multi-task Sequence Generation for Purchasing Decision Making in First-person Shooters

General AI Challenge - Round One: Gradual Learning

Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification

Learning an Explicit Hyperparameter Prediction Policy Conditioned on Tasks

Odds for the Brazilian 2018 president elections: An application of Bayesian statistics in contingency tables

Towards Meta-learned Algorithm Selection using Implicit Fidelity Information

Meta-learning from Learning Curves Challenge: Lessons learned from the First Round and Design of the Second Round

Related Research

NeurIPS'22 Cross-Domain MetaDL competition: Design and baseline results

Learning to Reason in Round-based Games: Multi-task Sequence Generation for Purchasing Decision Making in First-person Shooters

General AI Challenge - Round One: Gradual Learning

Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification

Learning an Explicit Hyperparameter Prediction Policy Conditioned on Tasks

Odds for the Brazilian 2018 president elections: An application of Bayesian statistics in contingency tables

Towards Meta-learned Algorithm Selection using Implicit Fidelity Information