Title :

DP-SGD with weight clipping

Author(s) :

Barczewski, Antoine [Auteur]
Machine Learning in Information Networks [MAGNET]
Ramon, Jan [Auteur]
Machine Learning in Information Networks [MAGNET]

Conference title :

CAp (Conférence sur l'Apprentissage automatique) 2024

Conference organizers(s) :

SSFAM (Société Savante Française d'Apprentissage Machine)
AFRIF (Association Française pour la Reconnaissance et l'Interprétation des Formes)

City :

Lille (France)

Country :

France

Start date of the conference :

2024-07-01

Publisher :

arXiv

Publication date :

2023

English keyword(s) :

Differential Privacy
Optimization
Machine Learning (cs.LG)

HAL domain(s) :

Informatique [cs]/Apprentissage [cs.LG]

English abstract : [en]

Recently, due to the popularity of deep neural networks and other methods whose training typically relies on the optimization of an objective function, and due to concerns for data privacy, there is a lot of interest in ...
Show more >Recently, due to the popularity of deep neural networks and other methods whose training typically relies on the optimization of an objective function, and due to concerns for data privacy, there is a lot of interest in differentially private gradient descent methods. To achieve differential privacy guarantees with a minimum amount of noise, it is important to be able to bound precisely the sensitivity of the information which the participants will observe. In this study, we present a novel approach that mitigates the bias arising from traditional gradient clipping. By leveraging a public upper bound of the Lipschitz value of the current model and its current location within the search domain, we can achieve refined noise level adjustments. We present a new algorithm with improved differential privacy guarantees and a systematic empirical evaluation, showing that our new approach outperforms existing approaches also in practice.Show less >

Language :

Anglais

Peer reviewed article :

Oui

Audience :

Internationale

Popular science :

Non

ANR Project :

Recherche en Medicine respectant la vie Privée

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL