• English
    • français
  • Help
  •  | 
  • Contact
  •  | 
  • About
  •  | 
  • Login
  • HAL portal
  •  | 
  • Pages Pro
  • EN
  •  / 
  • FR
View Item 
  •   LillOA Home
  • Liste des unités
  • Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
  • View Item
  •   LillOA Home
  • Liste des unités
  • Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Differentially Private Coordinate Descent ...
  • BibTeX
  • CSV
  • Excel
  • RIS

Document type :
Pré-publication ou Document de travail
Permalink :
http://hdl.handle.net/20.500.12210/57851
Title :
Differentially Private Coordinate Descent for Composite Empirical Risk Minimization
Author(s) :
Mangold, Paul [Auteur]
Machine Learning in Information Networks [MAGNET]
Bellet, Aurelien [Auteur] refId
Machine Learning in Information Networks [MAGNET]
Salmon, Joseph [Auteur]

Université de Montpellier [UM]
Institut Montpelliérain Alexander Grothendieck [IMAG]
Tommasi, Marc [Auteur] refId
Machine Learning in Information Networks [MAGNET]
HAL domain(s) :
Informatique [cs]/Apprentissage [cs.LG]
Statistiques [stat]/Machine Learning [stat.ML]
English abstract : [en]
Machine learning models can leak information about the data used to train them. Differentially Private (DP) variants of optimization algorithms like Stochastic Gradient Descent (DP-SGD) have been designed to mitigate this, ...
Show more >
Machine learning models can leak information about the data used to train them. Differentially Private (DP) variants of optimization algorithms like Stochastic Gradient Descent (DP-SGD) have been designed to mitigate this, inducing a trade-off between privacy and utility. In this paper, we propose a new method for composite Differentially Private Empirical Risk Minimization (DP-ERM): Differentially Private proximal Coordinate Descent (DP-CD). We analyze its utility through a novel theoretical analysis of inexact coordinate descent, and highlight some regimes where DP-CD outperforms DP-SGD, thanks to the possibility of using larger step sizes. We also prove new lower bounds for composite DP-ERM under coordinate-wise regularity assumptions, that are, in some settings, nearly matched by our algorithm. In practical implementations, the coordinate-wise nature of DP-CD updates demands special care in choosing the clipping thresholds used to bound individual contributions to the gradients. A natural parameterization of these thresholds emerges from our theory, limiting the addition of unnecessarily large noise without requiring coordinatewise hyperparameter tuning or extra computational cost.Show less >
Language :
Anglais
Collections :
  • Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Source :
Harvested from HAL
Submission date :
2021-11-17T02:01:42Z
Files
Thumbnail
  • https://hal.inria.fr/hal-03424974/document
  • Open access
  • Access the document
Université de Lille

Mentions légales
Université de Lille © 2017