Learning Landmark-Based Ensembles with ...
Document type :
Pré-publication ou Document de travail
Title :
Learning Landmark-Based Ensembles with Random Fourier Features and Gradient Boosting
Author(s) :
Gautheron, Léo [Auteur]
Laboratoire Hubert Curien [LabHC]
Germain, Pascal [Auteur]
MOdel for Data Analysis and Learning [MODAL]
Habrard, Amaury [Auteur]
Laboratoire Hubert Curien [LabHC]
Morvant, Emilie [Auteur]
Laboratoire Hubert Curien [LabHC]
Sebban, Marc [Auteur]
Laboratoire Hubert Curien [LabHC]
Zantedeschi, Valentina [Auteur]
Laboratoire Hubert Curien [LabHC]
Laboratoire Hubert Curien [LabHC]
Germain, Pascal [Auteur]
MOdel for Data Analysis and Learning [MODAL]
Habrard, Amaury [Auteur]
Laboratoire Hubert Curien [LabHC]
Morvant, Emilie [Auteur]
Laboratoire Hubert Curien [LabHC]
Sebban, Marc [Auteur]
Laboratoire Hubert Curien [LabHC]
Zantedeschi, Valentina [Auteur]
Laboratoire Hubert Curien [LabHC]
HAL domain(s) :
Statistiques [stat]/Machine Learning [stat.ML]
English abstract : [en]
We propose a Gradient Boosting algorithm for learning an ensemble of kernel functions adapted to the task at hand. Unlike state-of-the-art Multiple Kernel Learning techniques that make use of a pre-computed dictionary of ...
Show more >We propose a Gradient Boosting algorithm for learning an ensemble of kernel functions adapted to the task at hand. Unlike state-of-the-art Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it as a weighted sum of Random Fourier Features (RFF) and by optimizing their barycenter. This allows us to obtain a more versatile method, easier to setup and likely to have better performance. Our study builds on a recent result showing one can learn a kernel from RFF by computing the minimum of a PAC-Bayesian bound on the kernel alignment generalization loss, which is obtained efficiently from a closed-form solution. We conduct an experimental analysis to highlight the advantages of our method w.r.t. both Boosting-based and kernel-learning state-of-the-art methods.Show less >
Show more >We propose a Gradient Boosting algorithm for learning an ensemble of kernel functions adapted to the task at hand. Unlike state-of-the-art Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it as a weighted sum of Random Fourier Features (RFF) and by optimizing their barycenter. This allows us to obtain a more versatile method, easier to setup and likely to have better performance. Our study builds on a recent result showing one can learn a kernel from RFF by computing the minimum of a PAC-Bayesian bound on the kernel alignment generalization loss, which is obtained efficiently from a closed-form solution. We conduct an experimental analysis to highlight the advantages of our method w.r.t. both Boosting-based and kernel-learning state-of-the-art methods.Show less >
Language :
Anglais
Collections :
Source :
Files
- document
- Open access
- Access the document
- hal.pdf
- Open access
- Access the document
- 1906.06203
- Open access
- Access the document