Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound

Zantedeschi, Valentina; Viallard, Paul; Morvant, Emilie; Emonet, Rémi; Habrard, Amaury; Germain, Pascal; Guedj, Benjamin

Document type :

Autre communication scientifique (congrès sans actes - poster - séminaire...): Communication dans un congrès avec actes

Title :

Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound

Author(s) :

Zantedeschi, Valentina [Auteur]
MOdel for Data Analysis and Learning [MODAL]
The Inria London Programme [Inria-London]
Viallard, Paul [Auteur]
Laboratoire Hubert Curien [LabHC]
Morvant, Emilie [Auteur]
Laboratoire Hubert Curien [LabHC]
Emonet, Rémi [Auteur]
Laboratoire Hubert Curien [LabHC]
Habrard, Amaury [Auteur]
Laboratoire Hubert Curien [LabHC]
Germain, Pascal [Auteur]
Université Laval [Québec] [ULaval]
Guedj, Benjamin [Auteur]

Department of Computer science [University College of London] [UCL-CS]
Inria-CWI [Inria-CWI]
MOdel for Data Analysis and Learning [MODAL]

Conference title :

CAp 2022

City :

Vannes

Country :

France

Start date of the conference :

2022-07-05

HAL domain(s) :

Statistiques [stat]/Machine Learning [stat.ML]

English abstract : [en]

We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet ...
Show more >We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective. The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives - both with uninformed (data-independent) and informed (data-dependent) priors.Show less >

Language :

Anglais

Peer reviewed article :

Oui

Audience :

Nationale

Popular science :

Non

ANR Project :

Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations
Apprentissage PAC-bayésien agnostique

Collections :

Laboratoire Paul Painlevé - UMR 8524

Source :

Harvested from HAL

Learning Stochastic Majority Votes by ... BibTeX CSV Excel RIS

Learning Stochastic Majority Votes by ...

BibTeX

CSV

Excel

RIS