Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 2 appraisals
JMIR medical education
Training Empathetic Communication Skills in Medical Students With a Role-Prompted GPT-4o Chatbot: Quasi-Experimental Intervention Study
BACKGROUND: There is growing concern that artificial intelligence (AI) may diminish the quality of human relationships. However, in a context of widespread social importance (empathetic conversations between doctors and patients), AI can actually improve human conversational skills, potentially enhancing professional relationships. Recent advances in AI allow for realistically role-prompted counterparts for practicing professional conversations, enabling relational learning without the need for human counterparts. OBJECTIVE: This study aimed to show the effectiveness of AI chatbots for learning professional communicative skills in medical education. Specifically, we hypothesized that a single conversation with an AI chatbot improves communication skills in medical students across 4 different conversational competencies. METHODS: We conducted a quasi-experimental intervention study involving 4 distinct role-prompted scenarios (ie, shared decision-making, motivational interviewing, sexually transmitted diseases, and breaking bad news)-each designed to elicit in-depth empathic conversational skills aligned with key learning objectives in medical curricula. Students rated their competence for the 4 scenarios before and after a conversation with GPT-4o (OpenAI) using default settings, without fine-tuning. We expected higher perceived communication competence (PCC) in their conversation topic after the interaction compared with before the interaction in a 2-sided paired t test. Participants received AI-generated feedback, which they rated regarding adequacy. Post hoc analyses addressed gender and case effects, feedback adequacy, and prevalues in PCC. RESULTS: This study shows that a role-prompted GPT chatbot improves PCC in 162 medical students after a single conversation with mean of 13 (SD 4.8; 95% CI 12-14) prompt-response pairs. We found an increase in PCC with a mean difference of 0.94 (SD 1.64; 95% CI 0.69-1.20; Cohen d=0.58) from 5.89 (95% CI 5.55-6.23; scale 0-10) before the conversation to 6.83 (95% CI 6.55-7.12) after the conversation across 4 different patient role prompts. Furthermore, we found participants rating AI feedback of their conversation to be useful (mean 7.92, SD 1.61; 95% CI 7.67-8.17; scale 0-10), but feedback adequacy did not correspond to PCC increase (r=0.08; P=.32). CONCLUSIONS: Our results demonstrate how role-prompted GPT increases self-assessed communication competencies, introducing a novel tool for teaching relational learning. Our results present a starting point for using AI in education, particularly teaching communication in professional roles. On the basis of our findings in medical education, we anticipate further studies to investigate conversational training between lawyers and clients, marketers and customers, or managers and employees. Our research thus has implications for any field with a need for conversational training and relational learning.
14 June 2026
Read appraisal →JMIR medical education
Exploring the Feasibility of an Examiner-Worn Neck-Mounted Camera for Objective Structured Clinical Examination Assessment: Pilot Feasibility Study
BACKGROUND: The Objective Structured Clinical Examination (OSCE) is a prevalent method for evaluating clinical competence in medical education. As OSCEs become increasingly standardized and resource intensive, alternative evaluation methods are being explored, particularly because of the limited availability of certified examiners. However, few studies have investigated whether wearable technologies can support OSCE assessment. Wearable devices may provide a means of recording clinical skills from the examiner's perspective. OBJECTIVE: This pilot study, conducted in 2024, aimed to investigate the feasibility of using an examiner-worn neck-mounted camera for recording OSCE scenarios and to evaluate the evaluability of clinical performance from the recorded footage. METHODS: In total, 9 experienced medical educators participated in a simulated OSCE scenario involving electrocardiogram lead placement. All participants completed the initial live assessment and the postuse questionnaire, while 8 of 9 (89%) participants completed the subsequent video-based reassessment. Video recordings from both a fixed camera and a neck-mounted camera (THINKLET) were used to assess the evaluability of each OSCE item. Following a washout period, evaluators reassessed the neck-mounted camera recordings by using the original checklist, while fixed-camera recordings were used to judge the evaluability of each item. Agreement between live and video-based scoring was analyzed using percent agreement and the Cohen κ coefficient. A postevaluation questionnaire captured evaluators' experiences with the wearable device. RESULTS: Cohen κ ranged from 0.258 to 0.913 (mean 0.67, SD 0.20). Across checklist observations, more items were judged to be evaluable in the neck-mounted camera recordings than in the fixed-camera recordings, particularly for tasks requiring observation of fine motor skills. Evaluators reported generally positive experiences with the device, although some noted issues related to audio quality, comfort, posture restriction, and limited visibility at low angles. CONCLUSIONS: Although further investigation is needed, this pilot study suggests that an examiner-worn neck-mounted camera may be a valuable supplementary assessment tool for selected OSCE tasks. Further work is needed to refine the device, standardize recording protocols, and clarify how it can best support review and verification alongside live evaluation.
29 May 2026
Read appraisal →