An adaptive multi-agent system for task ...
Type de document :
Compte-rendu et recension critique d'ouvrage
Titre :
An adaptive multi-agent system for task reallocation in a MapReduce job
Auteur(s) :
Baert, Quentin [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Caron, Anne-Cecile [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
MORGE, Maxime [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Routier, Jean-Christophe [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Stathis, Kostas [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Caron, Anne-Cecile [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
MORGE, Maxime [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Routier, Jean-Christophe [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Systèmes Multi-Agents et Comportements [SMAC]
Stathis, Kostas [Auteur]
Titre de la revue :
Journal of Parallel and Distributed Computing
Pagination :
75-88
Éditeur :
Elsevier
Date de publication :
2021-07
ISSN :
0743-7315
Mot(s)-clé(s) en anglais :
MapReduce
BigData
Multi-agent systems
Negotiation
BigData
Multi-agent systems
Negotiation
Discipline(s) HAL :
Informatique [cs]/Système multi-agents [cs.MA]
Informatique [cs]/Modélisation et simulation
Informatique [cs]/Intelligence artificielle [cs.AI]
Informatique [cs]/Modélisation et simulation
Informatique [cs]/Intelligence artificielle [cs.AI]
Résumé en anglais : [en]
We study the problem of task reallocation for load-balancing of MapReduce jobs in applications that process large datasets. In this context, we propose a novel strategy based on cooperative agents used to optimise the task ...
Lire la suite >We study the problem of task reallocation for load-balancing of MapReduce jobs in applications that process large datasets. In this context, we propose a novel strategy based on cooperative agents used to optimise the task scheduling in a single MapReduce job. The novelty of our strategy lies in the ability of agents to identify opportunities within a current unbalanced allocation, which in turn trigger concurrent and one-to-many negotiations amongst agents to locally reallocate some of the tasks within a job. Our contribution is that tasks are reallocated according to the proximity of the resources and they are performed in accordance to the capabilities of the nodes in which agents are situated. To evaluate the adaptivity and responsiveness of our approach, we implement a prototype test-bed and conduct a vast panel of experiments in a heterogeneous environment and by exploring varying hardware configurations. This extensive experimentation reveals that our strategy significantly improves the overall runtime over the classical Hadoop data processing.Lire moins >
Lire la suite >We study the problem of task reallocation for load-balancing of MapReduce jobs in applications that process large datasets. In this context, we propose a novel strategy based on cooperative agents used to optimise the task scheduling in a single MapReduce job. The novelty of our strategy lies in the ability of agents to identify opportunities within a current unbalanced allocation, which in turn trigger concurrent and one-to-many negotiations amongst agents to locally reallocate some of the tasks within a job. Our contribution is that tasks are reallocated according to the proximity of the resources and they are performed in accordance to the capabilities of the nodes in which agents are situated. To evaluate the adaptivity and responsiveness of our approach, we implement a prototype test-bed and conduct a vast panel of experiments in a heterogeneous environment and by exploring varying hardware configurations. This extensive experimentation reveals that our strategy significantly improves the overall runtime over the classical Hadoop data processing.Lire moins >
Langue :
Anglais
Vulgarisation :
Non
Collections :
Source :
Fichiers
- https://hal.archives-ouvertes.fr/hal-03189190/document
- Accès libre
- Accéder au document
- https://hal.archives-ouvertes.fr/hal-03189190/document
- Accès libre
- Accéder au document
- https://hal.archives-ouvertes.fr/hal-03189190/document
- Accès libre
- Accéder au document
- baert20jpdc.pdf
- Accès libre
- Accéder au document
- baert20jpdc.pdf
- Accès libre
- Accéder au document
- document
- Accès libre
- Accéder au document