Differentially Private Coordinate Descent for Composite Empirical Risk Minimization

Mangold, Paul; Bellet, Aurelien; Salmon, Joseph; Tommasi, Marc

Type de document :

Pré-publication ou Document de travail

URL permanente :

http://hdl.handle.net/20.500.12210/57851

Titre :

Differentially Private Coordinate Descent for Composite Empirical Risk Minimization

Auteur(s) :

Mangold, Paul [Auteur]
Machine Learning in Information Networks [MAGNET]
Bellet, Aurelien [Auteur]

Machine Learning in Information Networks [MAGNET]
Salmon, Joseph [Auteur]
Centre National de la Recherche Scientifique [CNRS]
Université de Montpellier [UM]
Institut Montpelliérain Alexander Grothendieck [IMAG]
Tommasi, Marc [Auteur]

Machine Learning in Information Networks [MAGNET]

Discipline(s) HAL :

Informatique [cs]/Apprentissage [cs.LG]
Statistiques [stat]/Machine Learning [stat.ML]

Résumé en anglais : [en]

Machine learning models can leak information about the data used to train them. Differentially Private (DP) variants of optimization algorithms like Stochastic Gradient Descent (DP-SGD) have been designed to mitigate this, ...
Lire la suite >Machine learning models can leak information about the data used to train them. Differentially Private (DP) variants of optimization algorithms like Stochastic Gradient Descent (DP-SGD) have been designed to mitigate this, inducing a trade-off between privacy and utility. In this paper, we propose a new method for composite Differentially Private Empirical Risk Minimization (DP-ERM): Differentially Private proximal Coordinate Descent (DP-CD). We analyze its utility through a novel theoretical analysis of inexact coordinate descent, and highlight some regimes where DP-CD outperforms DP-SGD, thanks to the possibility of using larger step sizes. We also prove new lower bounds for composite DP-ERM under coordinate-wise regularity assumptions, that are, in some settings, nearly matched by our algorithm. In practical implementations, the coordinate-wise nature of DP-CD updates demands special care in choosing the clipping thresholds used to bound individual contributions to the gradients. A natural parameterization of these thresholds emerges from our theory, limiting the addition of unnecessarily large noise without requiring coordinatewise hyperparameter tuning or extra computational cost.Lire moins >

Langue :

Anglais

Collections :

Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL

Date de dépôt :

2021-11-17T02:01:42Z

Fichiers

https://hal.inria.fr/hal-03424974/document
Accès libre
Accéder au document

Differentially Private Coordinate Descent ... BibTeX CSV Excel RIS

Fichiers

Differentially Private Coordinate Descent ...

BibTeX

CSV

Excel

RIS