Meta-learning from Learning Curves: Challenge Design and Baseline Results

Nguyen, Manh Hung; Sun-Hosoya, Lisheng; Grinsztajn, Nathan; Guyon, Isabelle

Type de document :

Communication dans un congrès avec actes

DOI :

10.1109/IJCNN55064.2022.9892534

Titre :

Meta-learning from Learning Curves: Challenge Design and Baseline Results

Auteur(s) :

Nguyen, Manh Hung [Auteur]
Chalearn
Sun-Hosoya, Lisheng [Auteur]
Chalearn
Grinsztajn, Nathan [Auteur]
Université de Lille
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Scool [Scool]
Guyon, Isabelle [Auteur]
Chalearn
Laboratoire Interdisciplinaire des Sciences du Numérique [LISN]
Université Paris-Saclay
TAckling the Underspecified [TAU]

Titre de la manifestation scientifique :

IJCNN 2022 - International Joint Conference on Neural Networks

Ville :

Padua

Pays :

Italie

Date de début de la manifestation scientifique :

2022-07-18

Éditeur :

IEEE

Mot(s)-clé(s) en anglais :

AutoML
machine learning
meta-learning
learning curves
learning to learn
reinforcement learning

Discipline(s) HAL :

Informatique [cs]/Intelligence artificielle [cs.AI]

Résumé en anglais : [en]

Meta-learning has been widely studied and implemented in many Automated Machine Learning systems to improve the process of selecting and training Machine Learning models for new tasks, by leveraging expertise acquired on ...
Lire la suite >Meta-learning has been widely studied and implemented in many Automated Machine Learning systems to improve the process of selecting and training Machine Learning models for new tasks, by leveraging expertise acquired on previously observed tasks. We design a novel meta-learning challenge aiming at learning-to-learn from one of the most essential model evaluation data, the learning curve. It consists of multiple model evaluations collected during the process of training. A meta-learner is expected to apply a learned policy to learning curves of partially trained models on the task at hand, to rapidly find the best task solution, without training all potential models to convergence. This implies learning the exploration-exploitation trade-off. Our challenge is split into two phases: a development phase and a final test phase. In each phase, a meta-learner is meta-trained and meta-tested on validation learning curves (development phase) or test learning curves (final test phase). During meta-training, the meta-learner is allowed to learn from the provided learning curves in any possible way. In meta-testing, we borrowed the common Reinforcement Learning setting in which an agent (a meta-learner) learns by interacting with an environment storing pre-computed learning curves. A meta-learner must pay a cost (corresponding to the actual training and testing time) to reveal learning curve information progressively. The meta-learner is evaluated and ranked based on the average area under its learning curves. This challenge was accepted as part of the official selection of WCCI 2022 competitions.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Projet ANR :

Intelligence Artificielle pour Tous

Collections :