A simple dynamic bandit algorithm for hyper-parameter tuning

Shang, Xuedong; Kaufmann, Emilie; Valko, Michal

Document type :

Communication dans un congrès avec actes

Title :

A simple dynamic bandit algorithm for hyper-parameter tuning

Author(s) :

Shang, Xuedong [Auteur]
Sequential Learning [SEQUEL]
Kaufmann, Emilie [Auteur]

Sequential Learning [SEQUEL]
Valko, Michal [Auteur]

DeepMind [Paris]
Sequential Learning [SEQUEL]

Conference title :

Workshop on Automated Machine Learning at International Conference on Machine Learning

Conference organizers(s) :

AutoML@ICML 2019 - 6th ICML Workshop on Automated Machine Learning

City :

Long Beach

Country :

Etats-Unis d'Amérique

Start date of the conference :

2019-06-14

HAL domain(s) :

Statistiques [stat]/Machine Learning [stat.ML]

English abstract : [en]

Hyper-parameter tuning is a major part of modern machine learning systems. The tuning itself can be seen as a sequential resource allocation problem. As such, methods for multi-armed bandits have been already applied. In ...
Show more >Hyper-parameter tuning is a major part of modern machine learning systems. The tuning itself can be seen as a sequential resource allocation problem. As such, methods for multi-armed bandits have been already applied. In this paper, we view hyper-parameter optimization as an instance of best-arm identification in infinitely many-armed bandits. We propose D-TTTS, a new adaptive algorithm inspired by Thompson sampling, which dynamically balances between refining the estimate of the quality of hyper-parameter configurations previously explored and adding new hyper-parameter configurations to the pool of candidates. The algorithm is easy to implement and shows competitive performance compared to state-of-the-art algorithms for hyper-parameter tuning.Show less >

Language :

Anglais

Peer reviewed article :

Oui

Audience :

Internationale

Popular science :

Non

Collections :

Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL

Files

https://hal.inria.fr/hal-02145200/document
Open access
Access the document

https://hal.inria.fr/hal-02145200/document
Open access
Access the document

https://hal.inria.fr/hal-02145200/document
Open access
Access the document

document
Open access
Access the document

shang2019simple.pdf
Open access
Access the document

document
Open access
Access the document

shang2019simple.pdf
Open access
Access the document

A simple dynamic bandit algorithm for ... BibTeX CSV Excel RIS

Files

A simple dynamic bandit algorithm for ...

BibTeX

CSV

Excel

RIS