A single algorithm for both restless and rested rotting bandits

Seznec, Julien; Menard, Pierre; Lazaric, Alessandro; Valko, Michal

Type de document :

Communication dans un congrès avec actes

Titre :

A single algorithm for both restless and rested rotting bandits

Auteur(s) :

Seznec, Julien [Auteur]
Scool [Scool]
Lelivrescolaire.fr
Menard, Pierre [Auteur]
Scool [Scool]
Lazaric, Alessandro [Auteur]
Facebook AI Research [Paris] [FAIR]
Valko, Michal [Auteur]

DeepMind [Paris]

Titre de la manifestation scientifique :

International Conference on Artificial Intelligence and Statistics

Ville :

Palermo / Virtual

Pays :

Italie

Date de début de la manifestation scientifique :

2020-08-26

Discipline(s) HAL :

Statistiques [stat]/Machine Learning [stat.ML]

Résumé en anglais : [en]

In many application domains (e.g., recommender systems, intelligent tutoring systems), the rewards associated to the actions tend to decrease over time. This decay is either caused by the actions executed in the past (e.g., ...
Lire la suite >In many application domains (e.g., recommender systems, intelligent tutoring systems), the rewards associated to the actions tend to decrease over time. This decay is either caused by the actions executed in the past (e.g., a user may get bored when songs of the same genre are recommended over and over) or by an external factor (e.g., content becomes outdated). These two situations can be modeled as specific instances of the rested and restless bandit settings, where arms are rotting (i.e., their value decrease over time). These problems were thought to be significantly different, since Levine et al. (2017) showed that state-of-the-art algorithms for restless bandit perform poorly in the rested rotting setting. In this paper, we introduce a novel algorithm, Rotting Adaptive Window UCB (RAW-UCB), that achieves near-optimal regret in both rotting rested and restless bandit, without any prior knowledge of the setting (rested or restless) and the type of non-stationarity (e.g., piece-wise constant, bounded variation). This is in striking contrast with previous negative results showing that no algorithm can achieve similar results as soon as rewards are allowed to increase. We confirm our theoretical findings on a number of synthetic and datasetbased experiments.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Collections :

Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL

Fichiers

https://hal.inria.fr/hal-03287835/document
Accès libre
Accéder au document

https://hal.inria.fr/hal-03287835/document
Accès libre
Accéder au document

https://hal.inria.fr/hal-03287835/document
Accès libre
Accéder au document

https://hal.inria.fr/hal-03287835/document
Accès libre
Accéder au document

document
Accès libre
Accéder au document

seznec2020single.pdf
Accès libre
Accéder au document

document
Accès libre
Accéder au document

seznec2020single.pdf
Accès libre
Accéder au document

A single algorithm for both restless and ... BibTeX CSV Excel RIS

Fichiers

A single algorithm for both restless and ...

BibTeX

CSV

Excel

RIS