Tighter PAC-Bayes Generalisation Bounds ...
Document type :
Communication dans un congrès avec actes
Title :
Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty
Author(s) :
Biggs, Felix [Auteur]
Department of Computer science [University College of London] [UCL-CS]
Guedj, Benjamin [Auteur]
University College of London [London] [UCL]
Department of Computer science [University College of London] [UCL-CS]
Institut National de Recherche en Informatique et en Automatique [Inria]
Inria Lille - Nord Europe
The Alan Turing Institute
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Department of Computer science [University College of London] [UCL-CS]
Guedj, Benjamin [Auteur]
University College of London [London] [UCL]
Department of Computer science [University College of London] [UCL-CS]
Institut National de Recherche en Informatique et en Automatique [Inria]
Inria Lille - Nord Europe
The Alan Turing Institute
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Conference title :
AISTATS 2023 - 26th International Conference on Artificial Intelligence and Statistics
City :
Valencia
Country :
Espagne
Start date of the conference :
2023-04-25
Publication date :
2022-10-20
HAL domain(s) :
Informatique [cs]/Apprentissage [cs.LG]
English abstract : [en]
We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data ...
Show more >We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for $[-1, 1]$-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around $0$. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets.Show less >
Show more >We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for $[-1, 1]$-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around $0$. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets.Show less >
Language :
Anglais
Peer reviewed article :
Oui
Audience :
Internationale
Popular science :
Non
Comment :
22 pages
Collections :
Source :
Files
- 2210.11289
- Open access
- Access the document