Efficient preconditioned stochastic gradient ...
Document type :
Communication dans un congrès avec actes
Title :
Efficient preconditioned stochastic gradient descent for estimation in latent variable models
Author(s) :
Baey, Charlotte [Auteur correspondant]
Laboratoire Paul Painlevé - UMR 8524 [LPP]
Delattre, Maud [Auteur]
Mathématiques et Informatique Appliquées du Génome à l'Environnement [Jouy-En-Josas] [MaIAGE]
Kuhn, Estelle [Auteur]
Mathématiques et Informatique Appliquées du Génome à l'Environnement [Jouy-En-Josas] [MaIAGE]
Leger, Jean-Benoist [Auteur]
Heuristique et Diagnostic des Systèmes Complexes [Compiègne] [Heudiasyc]
Lemler, Sarah [Auteur]
Mathématiques et Informatique pour la Complexité et les Systèmes [MICS]
Laboratoire Paul Painlevé - UMR 8524 [LPP]
Delattre, Maud [Auteur]
Mathématiques et Informatique Appliquées du Génome à l'Environnement [Jouy-En-Josas] [MaIAGE]
Kuhn, Estelle [Auteur]
Mathématiques et Informatique Appliquées du Génome à l'Environnement [Jouy-En-Josas] [MaIAGE]
Leger, Jean-Benoist [Auteur]
Heuristique et Diagnostic des Systèmes Complexes [Compiègne] [Heudiasyc]
Lemler, Sarah [Auteur]
Mathématiques et Informatique pour la Complexité et les Systèmes [MICS]
Conference title :
40th International Conference on Machine Learning
City :
Honolulu (Hawaii), USA
Country :
Etats-Unis d'Amérique
Start date of the conference :
2023-07-23
Publication date :
2023-07
HAL domain(s) :
Mathématiques [math]
English abstract : [en]
Latent variable models are powerful tools for modeling complex phenomena involving in particular partially observed data, unobserved variables or underlying complex unknown structures. Inference is often difficult due to ...
Show more >Latent variable models are powerful tools for modeling complex phenomena involving in particular partially observed data, unobserved variables or underlying complex unknown structures. Inference is often difficult due to the latent structure of the model. To deal with parameter estimation in the presence of latent variables, well-known efficient methods exist, such as gradient-based and EM-type algorithms, but with practical and theoretical limitations. In this paper, we propose as an alternative for parameter estimation an efficient preconditioned stochastic gradient algorithm. Our method includes a preconditioning step based on a positive definite Fisher information matrix estimate. We prove convergence results for the proposed algorithm under mild assumptions for very general latent variables models. We illustrate through relevant simulations the performance of the proposed methodology in a nonlinear mixed effects model and in a stochastic block model.Show less >
Show more >Latent variable models are powerful tools for modeling complex phenomena involving in particular partially observed data, unobserved variables or underlying complex unknown structures. Inference is often difficult due to the latent structure of the model. To deal with parameter estimation in the presence of latent variables, well-known efficient methods exist, such as gradient-based and EM-type algorithms, but with practical and theoretical limitations. In this paper, we propose as an alternative for parameter estimation an efficient preconditioned stochastic gradient algorithm. Our method includes a preconditioning step based on a positive definite Fisher information matrix estimate. We prove convergence results for the proposed algorithm under mild assumptions for very general latent variables models. We illustrate through relevant simulations the performance of the proposed methodology in a nonlinear mixed effects model and in a stochastic block model.Show less >
Language :
Anglais
Peer reviewed article :
Oui
Audience :
Internationale
Popular science :
Non
Collections :
Source :
Files
- document
- Open access
- Access the document
- ICML2023_baey_et_al.pdf
- Open access
- Access the document