Supplementary file 1_Discrepancy between medical knowledge and clinical reasoning: evaluating large language models in prenatal diagnosis.docx

No significant differences were observed when stratified by modality (text-only, p = 0.146; imaging diagnostic questions, p = 0.603) or by any individual disease system (all p > 0.05). However, performance divergence emerged sharply across question types. In clinical vignette-based questions (n = 20), the chief physician (95.00%) significantly outperformed all other test-takers (p = 0.016), with a 25% gap over the best-performing LLM (Grok-4, 70.00%).

In sequential item sets, Grok-4 (95.74%) and the chief physician (93.62%) both significantly outperformed the attending physician (76.60%; both p < 0.05). Conversely, LLMs excelled in knowledge-retrieval tasks, with ChatGPT-5 and Claude-Sonnet-4.6 both achieving 100% accuracy in matching questions. No significant differences were detected among test-takers for factual recall, matching, or multiple-choice multiple-answer questions (all p > 0.05).</p>Conclusion<p>LLMs are well-positioned to serve as powerful knowledge augmentation tools, but are not yet ready to function as autonomous clinical reasoners in this complex, ethically consequential subspecialty.</p>