Confident but conflicting: Discordance among generative artificial intelligence systems in acute myeloid leukemia decision-making

Background

Generative artificial intelligence (AI) tools are increasingly used by clinicians to answer complex medical questions and support clinical decision-making. Their responses are often delivered with apparent confidence, potentially encouraging users to equate certainty or reproducibility with reliability. However, different AI systems may interpret the same clinical information differently. We evaluated whether commonly used generative AI systems provide consistent treatment recommendations, how frequently different systems agree, and whether disagreement is appropriately reflected by lower confidence.

Methods

We developed 20 clinical scenarios in acute myeloid leukemia spanning complex management decisions, including transplantation, measurable residual disease, post-remission therapy, germline evaluation, treatment following hypomethylating agent plus venetoclax failure, and relapsed disease. Four generative AI systems, ChatGPT (GPT-5.5 Instant), Claude (Sonnet 5), Gemini (3.5 Flash), and OpenEvidence, independently evaluated each scenario three times using standardized prompts in fresh sessions. Four hematologists independently reviewed the same scenarios. In total, 240 AI and 80 physician responses were analyzed. Outcomes included within-model reproducibility, pairwise agreement between AI systems, four-system concordance, physician agreement, and the frequency of conflicting AI recommendations accompanied by high reported confidence.

Results

Within-model reproducibility varied substantially across AI systems. Exact repeatability across three independent attempts was 65.0% for ChatGPT, 95.0% for Claude, 100.0% for Gemini, and 90.0% for OpenEvidence. Despite high reproducibility for several systems, agreement between different AI systems was considerably lower. Overall pairwise agreement was 69.2% (95% CI, 54.2% to 83.3%), and all four systems selected the same recommendation in only 50.0% of clinical scenarios. Hematologists reached consensus in 50.0% of scenarios, with unanimous agreement in 15.0%. Among the 10 scenarios in which AI systems disagreed, six contained at least one pair of conflicting recommendations for which both systems nevertheless reported high confidence.

Discussion & Conclusion

These findings demonstrate an important distinction between reproducibility and agreement in clinical AI. A system may repeatedly generate the same recommendation without that recommendation being shared by other AI systems evaluating identical clinical information. Importantly, disagreement was not consistently accompanied by lower confidence. Multiple systems could simultaneously express high confidence while recommending opposing management strategies. Confidence and reproducibility should therefore not be interpreted as proxies for clinical reliability. As AI tools become increasingly integrated into clinical medicine, clinicians should prioritize independent clinical validation rather than relying on apparent certainty or consistency. Cross-model confident disagreement may represent a simple, clinically meaningful warning signal identifying decisions that warrant heightened physician review.