Tighter PAC-Bayes Generalisation Bounds ...
Type de document :
Communication dans un congrès avec actes
Titre :
Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty
Auteur(s) :
Biggs, Felix [Auteur]
Department of Computer science [University College of London] [UCL-CS]
Guedj, Benjamin [Auteur]
University College of London [London] [UCL]
Department of Computer science [University College of London] [UCL-CS]
Institut National de Recherche en Informatique et en Automatique [Inria]
Inria Lille - Nord Europe
The Alan Turing Institute
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Department of Computer science [University College of London] [UCL-CS]
Guedj, Benjamin [Auteur]
University College of London [London] [UCL]
Department of Computer science [University College of London] [UCL-CS]
Institut National de Recherche en Informatique et en Automatique [Inria]
Inria Lille - Nord Europe
The Alan Turing Institute
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Titre de la manifestation scientifique :
AISTATS 2023 - 26th International Conference on Artificial Intelligence and Statistics
Ville :
Valencia
Pays :
Espagne
Date de début de la manifestation scientifique :
2023-04-25
Date de publication :
2022-10-20
Discipline(s) HAL :
Informatique [cs]/Apprentissage [cs.LG]
Résumé en anglais : [en]
We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data ...
Lire la suite >We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for $[-1, 1]$-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around $0$. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets.Lire moins >
Lire la suite >We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for $[-1, 1]$-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around $0$. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets.Lire moins >
Langue :
Anglais
Comité de lecture :
Oui
Audience :
Internationale
Vulgarisation :
Non
Commentaire :
22 pages
Collections :
Source :
Fichiers
- 2210.11289
- Accès libre
- Accéder au document