Comparison of the validity, perceived ...
Document type :
Article dans une revue scientifique: Article original
PMID :
Permalink :
Title :
Comparison of the validity, perceived usefulness, and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability.
Author(s) :
Marcilly, Romaric [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Zheng, Wu-Yi [Auteur]
Blackdog institute
Quindroit, Paul [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Pelayo, Sylvie [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Berdot, Sarah [Auteur]
Centre de Recherche des Cordeliers [CRC (UMR_S_1138 / U1138)]
Hôpital Européen Georges Pompidou [APHP] [HEGP]
Charpiat, Bruno [Auteur]
Hôpital de la Croix-Rousse [CHU - HCL]
Corny, Jennifer [Auteur]
Centre hospitalier Saint-Joseph [Paris]
Drouot, Sylvain [Auteur]
Frery, Pauline [Auteur]
CHR de Metz-Thionville
Leguelinel-Blache, Géraldine [Auteur]
Centre Hospitalier Universitaire de Nîmes [CHU Nîmes]
Centre Hospitalier Régional Universitaire [Montpellier] [CHRU Montpellier]
Mondet, Lisa [Auteur]
CHU Amiens-Picardie
Potier, Arnaud [Auteur]
Centre Hospitalier Régional Universitaire de Nancy [CHRU Nancy]
Centre hospitalier de Lunéville [GHEMM ]
Robert, Laurine [Auteur]
Centre Hospitalier Régional Universitaire [CHU Lille] [CHRU Lille]
Evaluation des technologies de santé et des pratiques médicales - ULR 2694 [METRICS]
Ferret, Laurie [Auteur]
Centre hospitalier [Valenciennes, Nord]
Baysari, Melissa [Auteur]
The University of Sydney
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Zheng, Wu-Yi [Auteur]
Blackdog institute
Quindroit, Paul [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Pelayo, Sylvie [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Berdot, Sarah [Auteur]
Centre de Recherche des Cordeliers [CRC (UMR_S_1138 / U1138)]
Hôpital Européen Georges Pompidou [APHP] [HEGP]
Charpiat, Bruno [Auteur]
Hôpital de la Croix-Rousse [CHU - HCL]
Corny, Jennifer [Auteur]
Centre hospitalier Saint-Joseph [Paris]
Drouot, Sylvain [Auteur]
Frery, Pauline [Auteur]
CHR de Metz-Thionville
Leguelinel-Blache, Géraldine [Auteur]
Centre Hospitalier Universitaire de Nîmes [CHU Nîmes]
Centre Hospitalier Régional Universitaire [Montpellier] [CHRU Montpellier]
Mondet, Lisa [Auteur]
CHU Amiens-Picardie
Potier, Arnaud [Auteur]
Centre Hospitalier Régional Universitaire de Nancy [CHRU Nancy]
Centre hospitalier de Lunéville [GHEMM ]
Robert, Laurine [Auteur]
Centre Hospitalier Régional Universitaire [CHU Lille] [CHRU Lille]
Evaluation des technologies de santé et des pratiques médicales - ULR 2694 [METRICS]
Ferret, Laurie [Auteur]
Centre hospitalier [Valenciennes, Nord]
Baysari, Melissa [Auteur]
The University of Sydney
Journal title :
International Journal of Medical Informatics
Abbreviated title :
Int J Med Inform
Volume number :
175
Pages :
105091
Publication date :
2023-05-15
ISSN :
1872-8243
English keyword(s) :
Assessment tool
clinical
Decision support systems
Human factors
Usability
Ergonomics
clinical
Decision support systems
Human factors
Usability
Ergonomics
HAL domain(s) :
Sciences du Vivant [q-bio]
English abstract : [en]
Objective: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts ...
Show more >Objective: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences. Method: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach. Results: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p <.001) and to compare the usability of alert systems during the procurement process (36 of 45; 0.8, p <.001). Conclusions: The TEMAS is perceived as more useful and is preferred by participants. The I-MeDeSA seems more relevant for quick evaluations that focus on the graphical user interface. The TEMAS seems to be more suitable for in-depth usability evaluations of alert systems. Even if both tools are perceived to be equally usable, they suffer from wording, instructional, and organizational problems that hinder their use. The results of this study will be used to improve the design of I-MeDeSA and TEMAS.Show less >
Show more >Objective: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences. Method: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach. Results: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p <.001) and to compare the usability of alert systems during the procurement process (36 of 45; 0.8, p <.001). Conclusions: The TEMAS is perceived as more useful and is preferred by participants. The I-MeDeSA seems more relevant for quick evaluations that focus on the graphical user interface. The TEMAS seems to be more suitable for in-depth usability evaluations of alert systems. Even if both tools are perceived to be equally usable, they suffer from wording, instructional, and organizational problems that hinder their use. The results of this study will be used to improve the design of I-MeDeSA and TEMAS.Show less >
Language :
Anglais
Audience :
Internationale
Popular science :
Non
Administrative institution(s) :
Université de Lille
CHU Lille
CHU Lille
Submission date :
2023-11-15T01:59:50Z
2024-04-02T11:44:28Z
2024-04-02T11:44:28Z