Title :

Measuring Exploration in Reinforcement Learning via Optimal Transport in Policy Space

Author(s) :

Nkhumise, Reabetswe M. [Auteur]
University of Sheffield [Sheffield]
Basu, Debabrota [Auteur]
Scool [Scool]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Inria Lille - Nord Europe
Université de Lille
Centrale Lille
Prescott, Tony J. [Auteur]
University of Sheffield [Sheffield]
Gilra, Aditya [Auteur]
Centrum Wiskunde & Informatica [CWI]

Publication date :

2024-02-14

HAL domain(s) :

Informatique [cs]/Apprentissage [cs.LG]
Informatique [cs]/Intelligence artificielle [cs.AI]
Mathématiques [math]/Systèmes dynamiques [math.DS]

English abstract : [en]

Exploration is the key ingredient of reinforcement learning (RL) that determines the speed and success of learning. Here, we quantify and compare the amount of exploration and learning accomplished by a Reinforcement ...
Show more >Exploration is the key ingredient of reinforcement learning (RL) that determines the speed and success of learning. Here, we quantify and compare the amount of exploration and learning accomplished by a Reinforcement Learning (RL) algorithm. Specifically, we propose a novel measure, named Exploration Index, that quantifies the relative effort of knowledge transfer (transferability) by an RL algorithm in comparison to supervised learning (SL) that transforms the initial data distribution of RL to the corresponding final data distribution. The comparison is established by formulating learning in RL as a sequence of SL tasks, and using optimal transport based metrics to compare the total path traversed by the RL and SL algorithms in the data distribution space. We perform extensive empirical analysis on various environments and with multiple algorithms to demonstrate that the exploration index yields insights about the exploration behaviour of any RL algorithm, and also allows us to compare the exploratory behaviours of different RL algorithms.Show less >

Language :

Anglais

ANR Project :

Causal eXplanations in Reinforcement Learning
Foundations of robustness and reliability in artificial intelligence
REPUBLIC: Vers l'IA responsable avec l'apprentissage par renforcement sous contraintes

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL