Data Sheet 3_Evaluation of a multi-component AI-guided mechanistic reasoning practical in experimental pharmacology: a single-group pre-post study.pdf

The intervention included a baseline assessment, an experimental logic mini-brief, an AI-supported mechanistic reasoning practical using a standardized reasoning prompt, an independent post-test, and a perception survey. The primary outcome was total assessment score change. Secondary outcomes included domain-specific scores, performance during the AI-supported practical, confidence, perceived AI helpfulness, student perceptions, and associations between practical performance and learning outcomes.

Results. Total assessment scores improved significantly from 76.82% ± 25.06% at baseline to 85.80% ± 21.27% after the intervention, with a mean gain of 8.98 ± 20.53 percentage points (Wilcoxon signed-rank test, Z = −2.597, p = 0.009, r = 0.39). Significant pre-post improvements were observed in coagulation pharmacology (p = 0.001, r = 0.49) and a smaller uncorrected improvement occurred in platelet pharmacology (p = 0.048, r = 0.30), while experimental reasoning scores increased numerically but did not reach statistical significance (p = 0.219, r = 0.19).

Among AI-supported practical completers (n = 43), the mean Step 3 reasoning score was 17.88 ± 4.01 out of 21 (85.1% ± 19.1%). Accuracy was highest for experimental interpretation (87.0%), followed by mechanism or pathway identification (85.0%) and drug-class identification (83.4%). Step 3 performance was positively associated with post-test score (Spearman’s rho = 0.606, p < 0.001).

The core perception and acceptability items showed excellent internal consistency (Cronbach’s alpha = 0.935), and student perceptions were generally favorable. Conclusion. A multi-component pharmacology practical containing an AI-guided reasoning prompt was feasible, acceptable, and associated with improved short-term assessment scores.

Given the single-group pre-post design, we cannot isolate the specific AI contribution from co-occurring instructional elements, such as the mini-brief, case-based practice, and retrieval-practice testing effects. Controlled studies are needed to assess causal efficacy and long-term retention.</p>