The Optimal Sample Size for Usability ...
Document type :
Article dans une revue scientifique: Article original
PMID :
Permalink :
Title :
The Optimal Sample Size for Usability Testing, From the Manufacturer's Perspective: A Value-of-Information Approach.
Author(s) :
Caron, Alexandre [Auteur]
Evaluation des technologies de santé et des pratiques médicales - ULR 2694 [METRICS]
Vandewalle, Vincent [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Marcilly, Romaric [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Rochat, Jessica [Auteur]
Dervaux, Benoit [Auteur]
Facteurs de risque et déterminants moléculaires des maladies liées au vieillissement (RID-AGE) - U1167
Evaluation des technologies de santé et des pratiques médicales - ULR 2694 [METRICS]
Vandewalle, Vincent [Auteur]
![refId](/themes/Mirage2//images/idref.png)
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Marcilly, Romaric [Auteur]
![refId](/themes/Mirage2//images/idref.png)
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Rochat, Jessica [Auteur]
Dervaux, Benoit [Auteur]
![refId](/themes/Mirage2//images/idref.png)
Facteurs de risque et déterminants moléculaires des maladies liées au vieillissement (RID-AGE) - U1167
Journal title :
Value in Health
Abbreviated title :
Value Health
Volume number :
25
Pages :
116-124
Publication date :
2022-01-17
ISSN :
1524-4733
English keyword(s) :
decision theory
early technology assessment
medical device
usability testing
value of information
early technology assessment
medical device
usability testing
value of information
HAL domain(s) :
Sciences du Vivant [q-bio]
English abstract : [en]
Objectives
For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device’s design. The manufacturer assesses the final ...
Show more >Objectives For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device’s design. The manufacturer assesses the final version of the device and determines the risk-benefit ratio for remaining errors. Nevertheless, the decision rule currently used to determine the sample size for this testing has statistical limitations and the lack of a clear decision-making perspective. Methods As an alternative, we developed a value-of-information analysis from the medical device manufacturer’s perspective. The consequences of use errors not detected during usability testing and the errors’ probability of occurrence were embedded in a loss function. The value of further testing was assessed as a reduction in the expected loss for the manufacturer. The optimal sample size was determined using the expected net benefit of sampling (ENBS) (the difference between the value provided by new participants and the cost of their inclusion). Results The value-of-information approach was applied to a real usability test of a needle-free adrenaline autoinjector. The initial estimate (performed on the first n = 20 participants) gave an optimal sample size of 100 participants and an ENBS of €255 453. This estimation was updated iteratively as new participants were included. After the inclusion of 90 participants, the ENBS was null for any sample size; hence, the cost of adding more participants outweighed the expected value of information, and therefore, the study could be stopped. Conclusions On the basis of these results, our method seems to be highly suitable for sample size estimation in the usability testing of medical devices before market access.Show less >
Show more >Objectives For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device’s design. The manufacturer assesses the final version of the device and determines the risk-benefit ratio for remaining errors. Nevertheless, the decision rule currently used to determine the sample size for this testing has statistical limitations and the lack of a clear decision-making perspective. Methods As an alternative, we developed a value-of-information analysis from the medical device manufacturer’s perspective. The consequences of use errors not detected during usability testing and the errors’ probability of occurrence were embedded in a loss function. The value of further testing was assessed as a reduction in the expected loss for the manufacturer. The optimal sample size was determined using the expected net benefit of sampling (ENBS) (the difference between the value provided by new participants and the cost of their inclusion). Results The value-of-information approach was applied to a real usability test of a needle-free adrenaline autoinjector. The initial estimate (performed on the first n = 20 participants) gave an optimal sample size of 100 participants and an ENBS of €255 453. This estimation was updated iteratively as new participants were included. After the inclusion of 90 participants, the ENBS was null for any sample size; hence, the cost of adding more participants outweighed the expected value of information, and therefore, the study could be stopped. Conclusions On the basis of these results, our method seems to be highly suitable for sample size estimation in the usability testing of medical devices before market access.Show less >
Language :
Anglais
Audience :
Internationale
Popular science :
Non
Administrative institution(s) :
Université de Lille
CHU Lille
CHU Lille
Collections :
Submission date :
2023-11-15T05:09:30Z
2024-05-13T09:03:03Z
2024-05-13T09:03:03Z
Files
- j.jval.2021.07.010
- Open access
- Access the document