Title :

PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales

Author(s) :

Haddouche, Maxime [Auteur]
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Guedj, Benjamin [Auteur]
MOdel for Data Analysis and Learning [MODAL]
The Inria London Programme [Inria-London]
The Alan Turing Institute
Department of Computer science [University College of London] [UCL-CS]
University College of London [London] [UCL]

Journal title :

Transactions on Machine Learning Research Journal

Publisher :

[Amherst Massachusetts]: OpenReview.net, 2022

Publication date :

2023-04

ISSN :

2835-8856

English keyword(s) :

PAC-Bayes
supermartingales
unbounded losses
generalisation bounds

HAL domain(s) :

Statistiques [stat]/Machine Learning [stat.ML]
Informatique [cs]/Apprentissage [cs.LG]
Statistiques [stat]/Théorie [stat.TH]

English abstract : [en]

While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e.g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted ...
Show more >While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e.g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted a growing interest in recent years. We contribute PAC-Bayes generalisation bounds for heavy-tailed losses under the sole assumption of bounded variance of the loss function. Under that assumption, we extend previous results from \citet{kuzborskij2019efron}. Our key technical contribution is exploiting an extention of Markov's inequality for supermartingales. Our proof technique unifies and extends different PAC-Bayesian frameworks by providing bounds for unbounded martingales as well as bounds for batch and online learning with heavy-tailed losses.Show less >

Language :

Anglais

Popular science :

Non

Collections :

• Laboratoire Paul Painlevé - UMR 8524

Source :

Harvested from HAL