Federated Multi-Task Learning under a ...
Type de document :
Communication dans un congrès avec actes
Titre :
Federated Multi-Task Learning under a Mixture of Distributions
Auteur(s) :
Marfoq, Othmane [Auteur]
Network Engineering and Operations [NEO ]
Accenture Labs [Sophia Antipolis]
Université Côte d'Azur [UniCA]
Neglia, Giovanni [Auteur]
Network Engineering and Operations [NEO ]
Université Côte d'Azur [UniCA]
Bellet, Aurelien [Auteur]
Machine Learning in Information Networks [MAGNET]
Kameni, Laetitia [Auteur]
Accenture Labs [Sophia Antipolis]
Vidal, Richard [Auteur]
Accenture Labs [Sophia Antipolis]
Network Engineering and Operations [NEO ]
Accenture Labs [Sophia Antipolis]
Université Côte d'Azur [UniCA]
Neglia, Giovanni [Auteur]
Network Engineering and Operations [NEO ]
Université Côte d'Azur [UniCA]
Bellet, Aurelien [Auteur]
Machine Learning in Information Networks [MAGNET]
Kameni, Laetitia [Auteur]
Accenture Labs [Sophia Antipolis]
Vidal, Richard [Auteur]
Accenture Labs [Sophia Antipolis]
Titre de la manifestation scientifique :
NeurIPS 2021 - 35th Conference on Neural Information Processing Systems
Ville :
Sydney / Virtual
Pays :
Australie
Date de début de la manifestation scientifique :
2021-12-06
Discipline(s) HAL :
Informatique [cs]/Apprentissage [cs.LG]
Informatique [cs]/Intelligence artificielle [cs.AI]
Mathématiques [math]/Optimisation et contrôle [math.OC]
Statistiques [stat]/Machine Learning [stat.ML]
Informatique [cs]/Intelligence artificielle [cs.AI]
Mathématiques [math]/Optimisation et contrôle [math.OC]
Statistiques [stat]/Machine Learning [stat.ML]
Résumé en anglais : [en]
The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models. First efforts in FL ...
Lire la suite >The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models. First efforts in FL focused on learning a single global model with good average performance across clients, but the global model may be arbitrarily bad for a given client, due to the inherent heterogeneity of local data distributions. Federated multi-task learning (MTL) approaches can learn personalized models by formulating an opportune penalized optimization problem. The penalization term can capture complex relations among personalized models, but eschews clear statistical assumptions about local data distributions. In this work, we propose to study federated MTL under the flexible assumption that each local data distribution is a mixture of unknown underlying distributions. This assumption encompasses most of the existing personalized FL approaches and leads to federated EM-like algorithms for both client-server and fully decentralized settings. Moreover, it provides a principled way to serve personalized models to clients not seen at training time. The algorithms' convergence is analyzed through a novel federated surrogate optimization framework, which can be of general interest. Experimental results on FL benchmarks show that our approach provides models with higher accuracy and fairness than state-of-the-art methods.Lire moins >
Lire la suite >The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models. First efforts in FL focused on learning a single global model with good average performance across clients, but the global model may be arbitrarily bad for a given client, due to the inherent heterogeneity of local data distributions. Federated multi-task learning (MTL) approaches can learn personalized models by formulating an opportune penalized optimization problem. The penalization term can capture complex relations among personalized models, but eschews clear statistical assumptions about local data distributions. In this work, we propose to study federated MTL under the flexible assumption that each local data distribution is a mixture of unknown underlying distributions. This assumption encompasses most of the existing personalized FL approaches and leads to federated EM-like algorithms for both client-server and fully decentralized settings. Moreover, it provides a principled way to serve personalized models to clients not seen at training time. The algorithms' convergence is analyzed through a novel federated surrogate optimization framework, which can be of general interest. Experimental results on FL benchmarks show that our approach provides models with higher accuracy and fairness than state-of-the-art methods.Lire moins >
Langue :
Anglais
Comité de lecture :
Oui
Audience :
Internationale
Vulgarisation :
Non
Projet ANR :
Commentaire :
77 pages, NeurIPS 2021
Collections :
Source :
Fichiers
- http://arxiv.org/pdf/2108.10252
- Accès libre
- Accéder au document
- https://hal.archives-ouvertes.fr/hal-03406994/document
- Accès libre
- Accéder au document
- https://hal.archives-ouvertes.fr/hal-03406994/document
- Accès libre
- Accéder au document
- document
- Accès libre
- Accéder au document
- 2108.10252.pdf
- Accès libre
- Accéder au document
- 2108.10252
- Accès libre
- Accéder au document