Designing Formulae for Ranking Search ...
Document type :
Article dans une revue scientifique: Article original
DOI :
PMID :
Permalink :
Title :
Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study.
Author(s) :
Douze, Laura [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Institut Coeur Poumon [CHU Lille]
Pelayo, Sylvie [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Messaadi, Nassir [Auteur]
425779|||Centre Hospitalier Régional Universitaire [CHU Lille] [CHRU Lille] (VALID)
Grosjean, Julien [Auteur]
Laboratoire d'Informatique Médicale et Ingénierie des Connaissances en e-Santé [LIMICS]
Kerdelhué, Gaétan [Auteur]
Laboratoire d'Informatique Médicale et Ingénierie des Connaissances en e-Santé [LIMICS]
Marcilly, Romaric [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Centre d'Investigation Clinique - Innovation Technologique de Lille - CIC 1403 - CIC 9301 [CIC Lille]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Institut Coeur Poumon [CHU Lille]
Pelayo, Sylvie [Auteur]
METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Messaadi, Nassir [Auteur]

425779|||Centre Hospitalier Régional Universitaire [CHU Lille] [CHRU Lille] (VALID)
Grosjean, Julien [Auteur]
Laboratoire d'Informatique Médicale et Ingénierie des Connaissances en e-Santé [LIMICS]
Kerdelhué, Gaétan [Auteur]
Laboratoire d'Informatique Médicale et Ingénierie des Connaissances en e-Santé [LIMICS]
Marcilly, Romaric [Auteur]

METRICS : Evaluation des technologies de santé et des pratiques médicales - ULR 2694
Centre d'Investigation Clinique - Innovation Technologique de Lille - CIC 1403 - CIC 9301 [CIC Lille]
Journal title :
JMIR Human Factors
Abbreviated title :
JMIR Hum Factors
Volume number :
9
Pages :
e30258
Publisher :
JMIR publications
Publication date :
2022-03-30
ISSN :
2292-9495
English keyword(s) :
information retrieval
search engine
topical relevance
search result ranking
user testing
human factors
search engine
topical relevance
search result ranking
user testing
human factors
HAL domain(s) :
Sciences du Vivant [q-bio]
English abstract : [en]
A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula ...
Show more >A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users. Objective: This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users. Methods: To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion. Results: A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article’s relevance, they judge its topic, methods, and value in clinical practice. Conclusions: Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine.Show less >
Show more >A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users. Objective: This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users. Methods: To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion. Results: A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article’s relevance, they judge its topic, methods, and value in clinical practice. Conclusions: Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine.Show less >
Language :
Anglais
Audience :
Internationale
Popular science :
Non
Administrative institution(s) :
Université de Lille
CHU Lille
CHU Lille
Submission date :
2023-11-15T04:25:47Z
2024-03-20T10:01:27Z
2024-04-05T06:31:55Z
2024-03-20T10:01:27Z
2024-04-05T06:31:55Z
Files
- document.pdf
- Non spécifié
- Open access
- Access the document