Practical Online Debugging of Spark-like ...
Type de document :
Communication dans un congrès avec actes
Titre :
Practical Online Debugging of Spark-like applications
Auteur(s) :
Marra, Matteo [Auteur]
Software Languages Lab [SLL]
Polito, Guillermo [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Analyses and Languages Constructs for Object-Oriented Application Evolution [RMOD]
Gonzalez Boix, Elisa [Auteur]
Software Languages Lab [SLL]
Software Languages Lab [SLL]
Polito, Guillermo [Auteur]
Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 [CRIStAL]
Analyses and Languages Constructs for Object-Oriented Application Evolution [RMOD]
Gonzalez Boix, Elisa [Auteur]
Software Languages Lab [SLL]
Titre de la manifestation scientifique :
IEEE QRS 2021 : International Conference on Software Security and Reliability
Ville :
Hainan Island
Pays :
Chine
Date de début de la manifestation scientifique :
2021-12-06
Titre de la revue :
IEEE QRS 2021 : International Conference on Software Security and Reliability
Discipline(s) HAL :
Informatique [cs]/Langage de programmation [cs.PL]
Résumé en anglais : [en]
Apache Spark is a framework widely used for writing Big Data analytics applications that offers a scalable and fault-tolerant model based on rescheduling failing tasks on other nodes. While this is well-suited for hardware ...
Lire la suite >Apache Spark is a framework widely used for writing Big Data analytics applications that offers a scalable and fault-tolerant model based on rescheduling failing tasks on other nodes. While this is well-suited for hardware and infrastructure errors, it is not for application errors as they will reappear in the rescheduled tasks. As a result, applications are killed, losing all the progress and forcing developers to restart them from scratch. Despite the popularity of such a failure-recovery model, understanding and debugging Sparklike applications remain challenging. When an error occurs, developers need to analyze huge log files or undergo timeconsuming replays to find the bug. To address these concerns, we present an online debugging approach tailored to Big Data analytics applications. Our approach includes local debugging of remote parallel exceptions through dynamic local checkpoints, extended with domain-specific debugging operations and live code updating functionality. To deal with data-cleaning errors, we extend our model to easily allow developers to automatically ignore exceptions that happen at runtime. We validate our solution through performance benchmarks that show how our debugging approach is comparable or better than state-of-theart debugging solutions for Big Data. Furthermore, we conduct a user study to compare our approach with another state-of-theart debugging approach, and results show a lower time to find the solution to a bug using our approach, as well as a generally good perception of the features of the debugger.Lire moins >
Lire la suite >Apache Spark is a framework widely used for writing Big Data analytics applications that offers a scalable and fault-tolerant model based on rescheduling failing tasks on other nodes. While this is well-suited for hardware and infrastructure errors, it is not for application errors as they will reappear in the rescheduled tasks. As a result, applications are killed, losing all the progress and forcing developers to restart them from scratch. Despite the popularity of such a failure-recovery model, understanding and debugging Sparklike applications remain challenging. When an error occurs, developers need to analyze huge log files or undergo timeconsuming replays to find the bug. To address these concerns, we present an online debugging approach tailored to Big Data analytics applications. Our approach includes local debugging of remote parallel exceptions through dynamic local checkpoints, extended with domain-specific debugging operations and live code updating functionality. To deal with data-cleaning errors, we extend our model to easily allow developers to automatically ignore exceptions that happen at runtime. We validate our solution through performance benchmarks that show how our debugging approach is comparable or better than state-of-theart debugging solutions for Big Data. Furthermore, we conduct a user study to compare our approach with another state-of-theart debugging approach, and results show a lower time to find the solution to a bug using our approach, as well as a generally good perception of the features of the debugger.Lire moins >
Langue :
Anglais
Comité de lecture :
Oui
Audience :
Internationale
Vulgarisation :
Non
Collections :
Source :
Fichiers
- https://hal.archives-ouvertes.fr/hal-03398772/document
- Accès libre
- Accéder au document
- https://hal.archives-ouvertes.fr/hal-03398772/document
- Accès libre
- Accéder au document
- https://hal.archives-ouvertes.fr/hal-03398772/document
- Accès libre
- Accéder au document
- document
- Accès libre
- Accéder au document
- paper.pdf
- Accès libre
- Accéder au document