AI-simulated clinical consultations: Assessing the potential of ChatGPT to support medical training

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Not reported quantitatively in abstract

• Primary Outcome: ChatGPT patient agent judged to be true to character most of the time and effective at expressing emotion; doctor agent rated as an effective communicator but occasionally used jargon; both agents produced repetitive responses undermining realism

• Nnt Or Sensitivity: Examiner agent showed 'good correlation' with human clinician marking — specific correlation coefficient not reported in abstract; moderate support reported for educational utility of simulated interactions

• Confidence Interval: Not reported

Clinical Application

Technically feasible as a supplementary tool given widespread access to large language model platforms; however, significant prompt engineering expertise is required to configure role-specific agents, and the written-only format limits ecological validity for clinical communication training. Institutional governance, data privacy, and quality assurance frameworks would need to be established before deployment in formal curricula. Australian medical education relies heavily on standardised patient programmes coordinated through medical schools and the Australian Medical Council (AMC) clinical examination pathway. The AMC clinical examination assesses communication, clinical reasoning, and procedural skills in a structured OSCE format. A scalable AI simulation adjunct could theoretically support preparation for AMC clinical examinations, particularly for international medical graduates (IMGs) who face access barriers to formal OSCE preparation programmes. However, the TGA does not currently regulate educational simulation software, and RACGP and RACP training programmes would need to independently validate any such tool before integration. The written-only interaction format is a significant limitation given that Australian OSCE assessments are conducted verbally. Data sovereignty and privacy considerations under the Australian Privacy Act 1988 would apply to any cloud-based AI platform used in medical education. Medical educators and postgraduate training programmes seeking scalable, low-cost adjuncts to standardised patient OSCE preparation, particularly in paediatrics

Abstract

BACKGROUND: Simulated medical scenarios are useful for evaluating and developing clinical competencies but scheduling them is expensive and time-consuming. Large language models show promise in role-playing tasks. We investigated the fidelity with which ChatGPT can mimic patients, clinicians and examiners in educational settings. OBJECTIVE: To determine the realism with which ChatGPT can portray patient, doctor and examiner roles, and the utility of these agents in clinical education. METHOD: We selected four paediatric scenarios from mock objective structured clinical examinations (OSCEs) and set up separate patient, doctor and examiner ChatGPT agents for each. The patient and doctor agents conversed with each other in written format. The examiner agent marked the doctor agent based on this conversation. Patients and clinicians familiar with the OSCE assessed the dialogues. RESULTS: The patient agent was judged to be true to character most of the time and good at expressing emotion. The doctor agent was reported to be an effective communicator but occasionally used jargon. Both agents tended to produce repetitive responses which undermined realism. The examiner agent had good correlation with human clinicians. There was moderate support for using the simulated interactions for educational purposes. CONCLUSION: Although the realism of the agents can be improved, ChatGPT can generate plausible proxies of participants in medical scenarios and could be useful for complementing standardised patient-based training.

References

1. 1.Saggar, A., Dimitrova, V., Sarikaya, D., Hogg, D., & Darling, J. C. (2026). AI-simulated clinical consultations: Assessing the potential of ChatGPT to support medical training. Archives of Disease in Childhood. https://doi.org/10.1136/archdischild-2025-329846