Titre :

Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisation

Auteur(s) :

Haddouche, Maxime [Auteur]
MOdel for Data Analysis and Learning [MODAL]
The Inria London Programme [Inria-London]
Guedj, Benjamin [Auteur]
MOdel for Data Analysis and Learning [MODAL]
The Inria London Programme [Inria-London]
The Alan Turing Institute
Department of Computer science [University College of London] [UCL-CS]
University College of London [London] [UCL]

Date de publication :

2023-04-14

Discipline(s) HAL :

Statistiques [stat]/Machine Learning [stat.ML]
Informatique [cs]/Apprentissage [cs.LG]

Résumé en anglais : [en]

PAC-Bayes learning is an established framework to both assess the generalisation ability of learning algorithms, and design new learning algorithm by exploiting generalisation bounds as training objectives. Most of the ...
Lire la suite >PAC-Bayes learning is an established framework to both assess the generalisation ability of learning algorithms, and design new learning algorithm by exploiting generalisation bounds as training objectives. Most of the exisiting bounds involve a \emph{Kullback-Leibler} (KL) divergence, which fails to capture the geometric properties of the loss function which are often useful in optimisation. We address this by extending the emerging \emph{Wasserstein PAC-Bayes} theory. We develop new PAC-Bayes bounds with Wasserstein distances replacing the usual KL, and demonstrate that sound optimisation guarantees translate to good generalisation abilities. In particular we provide generalisation bounds for the \emph{Bures-Wasserstein SGD} by exploiting its optimisation properties.Lire moins >

Langue :

Anglais

Collections :

• Laboratoire Paul Painlevé - UMR 8524

Source :

Harvested from HAL