Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 2 appraisals
JMIR research protocols
Coconstructing CHAMP, an Artificial Intelligence Chatbot for Pediatric Infectious Symptoms Management: Protocol for a Multiphase Participatory Study
BACKGROUND: Acute infectious symptoms are a leading cause of pediatric emergency department visits in Canada, many of which are low acuity and could be safely managed at home. Artificial intelligence (AI) chatbots offer a promising avenue for delivering accessible, evidence-based guidance to support families in managing these symptoms. OBJECTIVE: This study aims to adapt and coconstruct CHAMP (CHatbot to Assist the Management of Pediatric patients), an AI chatbot to support patients and families with acute pediatric infectious symptoms. CHAMP aims to deliver timely, tailored, and validated health information to support safe at-home self-management and informed care-seeking. METHODS: This multiphase, mixed methods participatory study will be conducted at the Montreal Children's Hospital in Montreal, Quebec, Canada. A coconstruction committee comprised of youth, parents, caregivers, and partners will be engaged as coresearchers. Eligible participants will include (1) youth aged 14-17 years and (2) parents and caregivers of children aged 0-17 years. The study comprises 5 phases. Phase 1 involves a qualitative needs assessment using focus groups with 20 participants to explore their informational needs, preferences, and concerns regarding pediatric infections and the use of AI chatbots. Phase 2 focuses on coconstructing and validating CHAMP's knowledge database through 3-5 workshops. Coresearchers will review pediatric clinical guidelines, map care questions and decision-making processes, and shape CHAMP's conversational framework. Phase 3 consists of iterative prototyping and testing through 3-5 workshops. Coresearchers will engage in prototyping and scenario testing, alongside preliminary usability and acceptability assessments. Phase 4 examines equity and accessibility through focus groups with 20 participants at risk of digital exclusion, as well as multilingual evaluation of an automated large language model-based translation layer. Phase 5 uses collaborative ethnography to explore the process of participatory coconstruction and its impact on CHAMP's design. RESULTS: Funding was secured in 2024, and Research Ethics Board approval was obtained in December 2024. As of December 2025, the coconstruction committee is being assembled, and Phase 1 recruitment is underway. CONCLUSIONS: This study will produce a functioning CHAMP prototype grounded in participatory, equitable, and responsible pediatric AI development. Findings will inform usability testing and an implementation-effectiveness evaluation, contributing to best practices for pediatric-centered AI health tools. By providing timely, tailored, and validated health information on acute infections, CHAMP may support safe at-home self-management, reduce preventable emergency department visits, ensure at-risk children are directed to appropriate care, and improve patient and family health care experiences.
28 May 2026
Read appraisal →European journal of pediatrics
Diagnostic accuracy of artificial intelligence versus 263 pediatric clinicians for childhood exanthems
Pediatric exanthematous diseases pose diagnostic challenges because clinical presentations overlap. To determine whether current artificial intelligence (AI) models achieve diagnostic accuracy within or above the performance distribution of pediatric residents and specialists for common rash-associated diseases. Participants and AI models were evaluated against definitive diagnoses confirmed by clinical features, laboratory findings, and consensus of two pediatric infectious disease specialists. A volunteer sample of 263 pediatric clinicians: 107 residents (years 1 through 4) and 156 specialists. Each clinician completed a blinded multiple-choice questionnaire with a clinical photograph and accompanying clinical data per case. The same cases were presented to three AI models: ChatGPT, Gemini, and Copilot. Among 263 clinicians (107 residents, 156 specialists), specialists scored higher than residents (median, 46 [IQR, 42-50] vs 41 [IQR, 36-46]; P < .001; r = 0.32). ChatGPT correctly diagnosed 53 of 61 cases (86.9%), Gemini 50 (82.0%), and Copilot 44 (72.1%). Both ChatGPT and Gemini exceeded the upper bound of the specialist population median 95% CI (47.17). All three AI models scored above the resident 95% CI upper bound (42.76). Disease-level accuracy ranged from 0% (insect bites, all models) to 100% (9 conditions, all models). Fourth-year residents scored higher than first- and second-year residents (P = .001; ε2 = 0.13).Conclusions: AI models given clinical data alongside images matched or exceeded specialist-level performance for pediatric exanthems. Accuracy varied by disease; failures clustered in conditions that require contextual reasoning. Physician oversight remains necessary where AI accuracy is lowest. What is Known • Childhood exanthematous diseases pose significant diagnostic challenges due to their overlapping clinical presentations. • Artificial intelligence models are becoming increasingly proficient in accurately diagnosing these conditions. What is New • In this diagnostic accuracy study of 61 cases evaluated by 263 clinicians and 3 artificial intelligence models, ChatGPT (86.9%) and Gemini (82.0%) exceeded the 95% CI upper bound of the specialist population median. Disease-level accuracy ranged from 0 to 100% across models • Artificial intelligence models given clinical data alongside images can match or exceed specialist-level accuracy for common pediatric exanthems, but failures in context-dependent diagnoses require physician oversight.
12 May 2026
Read appraisal →