How good is PAC-Bayes at explaining ...
Document type :
Pré-publication ou Document de travail
Permalink :
Title :
How good is PAC-Bayes at explaining generalisation?
Author(s) :
Picard-Weibel, Antoine [Auteur]
MOdel for Data Analysis and Learning [MODAL]
Inria Lille - Nord Europe
Centre International de Recherche Sur l'Eau et l'Environnement [Suez] [CIRSEE]
Clerico, Eugenio [Auteur]
Universitat Pompeu Fabra [Barcelona] [UPF]
Moscoviz, Roman [Auteur]
Centre International de Recherche Sur l'Eau et l'Environnement [Suez] [CIRSEE]
Guedj, Benjamin [Auteur]
University College of London [London] [UCL]
Inria Lille - Nord Europe
MOdel for Data Analysis and Learning [MODAL]
Inria Lille - Nord Europe
Centre International de Recherche Sur l'Eau et l'Environnement [Suez] [CIRSEE]
Clerico, Eugenio [Auteur]
Universitat Pompeu Fabra [Barcelona] [UPF]
Moscoviz, Roman [Auteur]
Centre International de Recherche Sur l'Eau et l'Environnement [Suez] [CIRSEE]
Guedj, Benjamin [Auteur]

University College of London [London] [UCL]
Inria Lille - Nord Europe
Publication date :
2025
English keyword(s) :
Machine Learning (stat.ML)
Machine Learning (cs.LG)
FOS: Computer and information sciences
Machine Learning (cs.LG)
FOS: Computer and information sciences
HAL domain(s) :
Informatique [cs]/Apprentissage [cs.LG]
Mathématiques [math]/Statistiques [math.ST]
Mathématiques [math]/Statistiques [math.ST]
English abstract : [en]
We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced ...
Show more >We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced by the prior distribution. In particular, achieving a target generalisation level is only achievable if the prior places sufficient mass on high-performing predictors. We relate these requirements to the prevalent practice of using data-dependent priors in deep learning PAC-Bayes applications, and discuss the implications for the claim that PAC-Bayes ``explains'' generalisation.Show less >
Show more >We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced by the prior distribution. In particular, achieving a target generalisation level is only achievable if the prior places sufficient mass on high-performing predictors. We relate these requirements to the prevalent practice of using data-dependent priors in deep learning PAC-Bayes applications, and discuss the implications for the claim that PAC-Bayes ``explains'' generalisation.Show less >
Language :
Anglais
Collections :
Source :
Submission date :
2025-03-19T07:28:47Z
Files
- 2503.08231
- Open access
- Access the document