Titre :

Minimax Excess Risk of First-Order Methods for Statistical Learning with Data-Dependent Oracles

Auteur(s) :

Scaman, Kevin [Auteur]
Département d'informatique - ENS-PSL [DI-ENS]
Apprentissage, graphes et optimisation distribuée [ARGO]
Even, Mathieu [Auteur]
Département d'informatique - ENS-PSL [DI-ENS]
Bars, Batiste Le [Auteur]
Machine Learning in Information Networks [MAGNET]
Massoulié, Laurent [Auteur]
Département d'informatique - ENS-PSL [DI-ENS]
Apprentissage, graphes et optimisation distribuée [ARGO]

Titre de la manifestation scientifique :

AISTATS 2024 - International Conference on Artificial Intelligence and Statistics

Ville :

Valencia

Pays :

Espagne

Date de début de la manifestation scientifique :

2024-05-02

Date de publication :

2024

Discipline(s) HAL :

Statistiques [stat]/Machine Learning [stat.ML]
Informatique [cs]/Intelligence artificielle [cs.AI]

Résumé en anglais : [en]

In this paper, our aim is to analyse the generalization capabilities of first-order methods for statistical learning in multiple, different yet related, scenarios including supervised learning, transfer learning, robust ...
Lire la suite >In this paper, our aim is to analyse the generalization capabilities of first-order methods for statistical learning in multiple, different yet related, scenarios including supervised learning, transfer learning, robust learning and federated learning. To do so, we provide sharp upper and lower bounds for the minimax excess risk of strongly convex and smooth statistical learning when the gradient is accessed through partial observations given by a data-dependent oracle. This novel class of oracles can query the gradient with any given data distribution, and is thus well suited to scenarios in which the training data distribution does not match the target (or test) distribution. In particular, our upper and lower bounds are proportional to the smallest mean square error achievable by gradient estimators, thus allowing us to easily derive multiple sharp bounds in the aforementioned scenarios using the extensive literature on parameter estimation.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Projet ANR :

Resilient, Decentralized and Privacy-Preserving Machine Learning
PaRis Artificial Intelligence Research InstitutE

Commentaire :

22 pages, 0 figures

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL