Table 1_Performance of large language models in prescription review: a comparative study.xlsx

Quantitative comparisons were performed on overall detection rate, question type identification and adaptability across clinical specialties. Prescription-level repeated-measures analysis of variance (ANOVA) and Holm-adjusted pairwise paired t-tests were used to examine inter-model statistical differences.</p>Results<p>The five models yielded item-level prescription detection rate ranging from 45.19% to 70.37% across 270 standardized medication assessment items.

Their performance ranked from highest to lowest as follows: Kimi, DeepSeek, Qwen, Doubao and ChatGPT. Overall, the four Chinese domestic models included in this study outperformed the single tested international model (ChatGPT) in this China-focused prescription review task. Repeated-measures ANOVA confirmed statistically significant within-prescription differences in overall performance (F = 22.46, P < 0.001).

While all models performed well at identifying inappropriate dosage forms and administration routes, they were less effective in detecting issues involving absent allergy test documentation and flawed result interpretation. Prescription reviews for the respiratory department yielded the best results, whereas detection rate was substantially lower for specialized clinical specialties such as oncology, pediatrics and pain management.

Even the best-performing model presented a high false-negative rate and cannot yet satisfy the strict reliability criteria for clinical application in prescription review.</p>Conclusion<p>Performance varied markedly across the five LLMs in prescription review. Although these models can effectively identify routine and straightforward medication issues, they demonstrate prominent deficiencies in assessing complex prescriptions from specialist clinical specialties and detecting hidden medication risks.

Currently, LLMs are only suitable for preliminary auxiliary screening and cannot substitute dedicated prescription review software or manual pharmacist evaluation.</p>