A PAC-Bayesian Link Between Generalisation ...
Type de document :
Pré-publication ou Document de travail
Titre :
A PAC-Bayesian Link Between Generalisation and Flat Minima
Auteur(s) :
Haddouche, Maxime [Auteur]
Université de Lille
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Viallard, Paul [Auteur]
Université de Rennes [UR]
Large Scale Collaborative Data Mining [LACODAM]
Şimşekli, Umut [Auteur]
Statistical Machine Learning and Parsimony [SIERRA]
Guedj, Benjamin [Auteur]
University College of London [London] [UCL]
Department of Computer science [University College of London] [UCL-CS]
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
The Alan Turing Institute
Université de Lille
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
Viallard, Paul [Auteur]
Université de Rennes [UR]
Large Scale Collaborative Data Mining [LACODAM]
Şimşekli, Umut [Auteur]
Statistical Machine Learning and Parsimony [SIERRA]
Guedj, Benjamin [Auteur]
![refId](/themes/Mirage2//images/idref.png)
University College of London [London] [UCL]
Department of Computer science [University College of London] [UCL-CS]
The Inria London Programme [Inria-London]
MOdel for Data Analysis and Learning [MODAL]
The Alan Turing Institute
Mot(s)-clé(s) en anglais :
Generalisation Bounds
PAC-Bayes
Flat Minima
Poincaré
Log-Sobolev Inequalities
PAC-Bayes
Flat Minima
Poincaré
Log-Sobolev Inequalities
Discipline(s) HAL :
Statistiques [stat]
Informatique [cs]
Informatique [cs]
Résumé en anglais : [en]
Modern machine learning usually involves predictors in the overparametrised setting (number of trained parameters greater than dataset size), and their training yield not only good performances on training data, but also ...
Lire la suite >Modern machine learning usually involves predictors in the overparametrised setting (number of trained parameters greater than dataset size), and their training yield not only good performances on training data, but also good generalisation capacity. This phenomenon challenges many theoretical results, and remains an open problem. To reach a better understanding, we provide novel generalisation bounds involving gradient terms. To do so, we combine the PAC-Bayes toolbox with Poincaré and Log-Sobolev inequalities, avoiding an explicit dependency on dimension of the predictor space. Our results highlight the positive influence of flat minima (being minima with a neighbourhood nearly minimising the learning problem as well) on generalisation performances, involving directly the benefits of the optimisation phase.Lire moins >
Lire la suite >Modern machine learning usually involves predictors in the overparametrised setting (number of trained parameters greater than dataset size), and their training yield not only good performances on training data, but also good generalisation capacity. This phenomenon challenges many theoretical results, and remains an open problem. To reach a better understanding, we provide novel generalisation bounds involving gradient terms. To do so, we combine the PAC-Bayes toolbox with Poincaré and Log-Sobolev inequalities, avoiding an explicit dependency on dimension of the predictor space. Our results highlight the positive influence of flat minima (being minima with a neighbourhood nearly minimising the learning problem as well) on generalisation performances, involving directly the benefits of the optimisation phase.Lire moins >
Langue :
Anglais
Projet ANR :
Collections :
Source :
Fichiers
- document
- Accès libre
- Accéder au document
- PLS_inequalities_x_PAC_Bayes.pdf
- Accès libre
- Accéder au document