Titre :

A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm

Auteur(s) :

Zheng, Wenjie [Auteur]
Machine Learning and Information Access [MLIA]
Bellet, Aurelien [Auteur]
Machine Learning in Information Networks [MAGNET]
Gallinari, Patrick [Auteur]
Machine Learning and Information Access [MLIA]

Titre de la revue :

Machine Learning

Éditeur :

Springer Verlag

Date de publication :

2018

ISSN :

0885-6125

Mot(s)-clé(s) en anglais :

Frank–Wolfe algorithm
Low-rank learning
Trace norm
Distributed optimization
Multi-task learning
Multinomial logistic regression

Discipline(s) HAL :

Informatique [cs]/Apprentissage [cs.LG]
Statistiques [stat]/Machine Learning [stat.ML]

Résumé en anglais : [en]

We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, ...
Lire la suite >We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, a distributed Frank-Wolfe algorithm which leverages the low-rank structure of its updates to achieve efficiency in time, memory and communication usage. The step at the heart of DFW-Trace is solved approximately using a distributed version of the power method. We provide a theoretical analysis of the convergence of DFW-Trace, showing that we can ensure sublinear convergence in expectation to an optimal solution with few power iterations per epoch. We implement DFW-Trace in the Apache Spark distributed programming framework and validate the usefulness of our approach on synthetic and real data, including the ImageNet dataset with high-dimensional features extracted from a deep neural network.Lire moins >

Langue :

Anglais

Comité de lecture :

Oui

Audience :

Internationale

Vulgarisation :

Non

Projet ANR :

Apprentissage automatique décentralisé et personnalisé sous contraintes

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL