Evaluating LLMs as Pharmaceutical Care Decision Support Tools Across Multiple Case Scenarios

DOI

10.2991/978-94-6463-604-8_24How to use a DOI?

Keywords

Healthcare; Large Language Models; Pharmaceutical Care Decision-Making; Artificial Intelligence

Abstract

In the evolving landscape of healthcare, pharmacists face increasing challenges in providing accurate, reliable, and prompt patient care amidst growing complexity in clinical settings. The continuous advancement of diseases, pharmaceutical sciences, and treatment guidelines requires pharmacists to stay up-to-date. However, the real-world burden of non-clinical tasks often impedes this effort. Recent practice of Large Language Models (LLMs) offers promising potential to support pharmacists in their professional duties. This study aims to evaluate the capability of LLMs in assisting pharmacists with pharmaceutical care decision-making. Three pharmaceutical cases (hypertension, hyperlipidemia, and angina pectoris) and related guidelines were input into the LLM, and their responses were assessed through both subjective and objective evaluations. The results indicated that, despite our efforts, the LLM fell short of satisfactory performance in terms of accuracy and reasoning. It was evident that the LLM's outputs still required human supervision and could not be accepted without scrutiny. However, the experts agreed that the LLM would be beneficial as a reference tool and in facilitating faster decision-making. Future research will focus on improving LLM performance.

Copyright

© 2024 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Vania Amanda Samor
AU  - Muhammad Yeza Baihaqi
AU  - Edmun Halawa
AU  - Luh Rai Maduretno Asvinigita
AU  - Sarah Nabila Hakim
AU  - Mela Septi Rofika
PY  - 2024
DA  - 2024/12/19
TI  - Evaluating LLMs as Pharmaceutical Care Decision Support Tools Across Multiple Case Scenarios
BT  - Proceedings of the International Conference on Medical Science and Health (ICOMESH 2024)
PB  - Atlantis Press
SP  - 273
EP  - 282
SN  - 2468-5739
UR  - https://doi.org/10.2991/978-94-6463-604-8_24
DO  - 10.2991/978-94-6463-604-8_24
ID  - Samor2024
ER  -