A kernel-based approach to non-stationary reinforcement learning in metric spaces

Domingues, Omar; Ménard, Pierre; Pirotta, Matteo; Kaufmann, Emilie; Valko, Michal

Document type :

Communication dans un congrès avec actes

Title :

A kernel-based approach to non-stationary reinforcement learning in metric spaces

Author(s) :

Domingues, Omar [Auteur]
Scool [Scool]
Ménard, Pierre [Auteur]
Otto-von-Guericke-Universität Magdeburg = Otto-von-Guericke University [Magdeburg] [OVGU]
Pirotta, Matteo [Auteur]
Facebook AI Research [Paris] [FAIR]
Kaufmann, Emilie [Auteur]

Scool [Scool]
Valko, Michal [Auteur]

DeepMind [Paris]

Conference title :

International Conference on Artificial Intelligence and Statistics

City :

San Diego / Virtual

Country :

Etats-Unis d'Amérique

Start date of the conference :

2021-04-13

HAL domain(s) :

Statistiques [stat]/Machine Learning [stat.ML]

English abstract : [en]

In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in nonstationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP ...
Show more >In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in nonstationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.Show less >

Language :

Anglais

Peer reviewed article :

Oui

Audience :

Internationale

Popular science :

Non

European Project :

CHISTERA DELTA

Collections :