Titre :

Fair multi-agent task allocation for large datasets analysis

Auteur(s) :

Baert, Quentin [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Caron, Anne-Cecile [Auteur]
Systèmes Multi-Agents et Comportements [SMAC]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Morge, Maxime [Auteur]
Systèmes Multi-Agents et Comportements [SMAC]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Routier, Jean-Christophe [Auteur]
Systèmes Multi-Agents et Comportements [SMAC]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]

Titre de la revue :

Knowledge and Information Systems (KAIS)

Pagination :

591-615

Éditeur :

Springer

Date de publication :

2017-07-19

ISSN :

0219-1377

Mot(s)-clé(s) en anglais :

Big data
Negotiation
Multi-agent system
MapReduce

Discipline(s) HAL :

Informatique [cs]/Système multi-agents [cs.MA]
Informatique [cs]/Modélisation et simulation
Informatique [cs]/Intelligence artificielle [cs.AI]

Résumé en anglais : [en]

MapReduce is a design pattern for processing large datasets distributed on a cluster. Its performances are linked to the data structure and the runtime environment. Indeed, data skew can yield an unfair task allocation, ...
Lire la suite >MapReduce is a design pattern for processing large datasets distributed on a cluster. Its performances are linked to the data structure and the runtime environment. Indeed, data skew can yield an unfair task allocation, but even when the initial allocation produced by the partition function is well balanced, an unfair allocation can occur during the reduce phase due to the heterogeneous performance of nodes. For these reasons, we propose an adaptive multi-agent system. In our approach, the reducer agents interact during the job and the task reallocation is based on negotiation in order to decrease the workload of the most loaded reducer and so the runtime. In this paper, we propose and evaluate two negotiation strategies. Finally, we experiment our multi-agent system with real-world datasets over heterogeneous runtime environment.Lire moins >

Langue :

Anglais

Vulgarisation :

Non

Collections :

• Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189

Source :

Harvested from HAL