Titre :

Softened approximate policy iteration for Markov games

Auteur(s) :

Pérolat, Julien [Auteur]
Université de Lille, Sciences et Technologies
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Sequential Learning [SEQUEL]
Piot, Bilal [Auteur]
Université de Lille, Sciences et Technologies
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Sequential Learning [SEQUEL]
Geist, Matthieu [Auteur]
MAchine Learning and Interactive Systems [MALIS]
Scherrer, Bruno [Auteur]
Institut Élie Cartan de Lorraine [IECL]
Biology, genetics and statistics [BIGS]
Pietquin, Olivier [Auteur]
Institut universitaire de France [IUF]
Université de Lille, Sciences et Technologies
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Sequential Learning [SEQUEL]

Titre de la manifestation scientifique :

ICML 2016 - 33rd International Conference on Machine Learning

Ville :

New York City

Pays :

Etats-Unis d'Amérique

Date de début de la manifestation scientifique :

2016-06-19

Discipline(s) HAL :

Mathématiques [math]/Optimisation et contrôle [math.OC]
Computer Science [cs]/Operations Research [math.OC]
Informatique [cs]/Complexité [cs.CC]
Informatique [cs]/Apprentissage [cs.LG]
Mathématiques [math]/Statistiques [math.ST]

Résumé en anglais : [en]

This paper reports theoretical and empirical investigations on the use of quasi-Newton methods to minimize the Optimal Bellman Residual (OBR) of zero-sum two-player Markov Games. First, it reveals that state-of-the-art ...
Lire la suite >This paper reports theoretical and empirical investigations on the use of quasi-Newton methods to minimize the Optimal Bellman Residual (OBR) of zero-sum two-player Markov Games. First, it reveals that state-of-the-art algorithms can be derived by the direct application of New-ton's method to different norms of the OBR. More precisely, when applied to the norm of the OBR, Newton's method results in the Bellman Residual Minimization Policy Iteration (BRMPI) and, when applied to the norm of the Projected OBR (POBR), it results into the standard Least Squares Policy Iteration (LSPI) algorithm. Consequently , new algorithms are proposed, making use of quasi-Newton methods to minimize the OBR and the POBR so as to take benefit of enhanced empirical performances at low cost. Indeed , using a quasi-Newton method approach introduces slight modifications in term of coding of LSPI and BRMPI but improves significantly both the stability and the performance of those algorithms. These phenomena are illustrated on an experiment conducted on artificially constructed games called Garnets.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL