Evaluation of the quality, reliability, and readability of ChatGPT-4 responses related to the treatment and rehabilitation of children with cerebral palsy

Clinical Snapshot

Key Findings

• P Value: No statistically significant differences between question categories

• Effect Size: mDISCERN scores: median 3-4 (scale 1-5), GQS scores: median 3-3.5 (scale 1-5)

• Primary Outcome: ChatGPT-4 responses showed moderate reliability and quality for cerebral palsy information

• Nnt Or Sensitivity: Inter-rater reliability: ICC 0.851 (mDISCERN), ICC 0.824 (GQS)

• Confidence Interval: Not reported - descriptive study design

Clinical Application

AI tools increasingly accessible but require healthcare professional oversight Relevant to Australian families but cultural adaptation needed. Consider NDIS context for cerebral palsy support. TGA has no specific AI health information regulations yet Families seeking information about cerebral palsy treatment and rehabilitation

Abstract

UNLABELLED: This study aims to evaluate the reliability, quality, and readability of ChatGPT-4's responses to questions about cerebral palsy (CP) and rehabilitation strategies. The 56 most frequently asked questions by families about CP, its treatment, and rehabilitation strategies were divided into five categories (A1-5) and asked in ChatGPT-4. The reliability, quality, and readability of the responses were assessed by two researchers, respectively, using the modified DISCERN (mDISCERN) tool, the Global Quality Scale (GQS), and the Flesch Reading Ease Scale (FRE). Median (IQR) values for mDISCERN ranged from 3 (3-3) in A5 to 4 (3-4) in A1, while GQS values ranged from 3 (3-3.5) in A2 to 3.5 (3-4) in A1. The mean readability values assessed with FRE ranged from 31.47 ± 12.92 to 44.84 ± 17.39. No statistically significant differences were observed between the categories. The ICC values suggested very good agreement for both scales, with 0.851 for the mDISCERN total score and 0.824 for the GQS total score. CONCLUSION: This study suggests that ChatGPT-4 provides moderate reliability and generally acceptable quality of responses regarding CP treatment and rehabilitation. FRE scores indicated that many responses were difficult for families to understand. While ChatGPT-4 may serve as a supportive source of general information, its outputs should be interpreted with caution, particularly in clinical contexts, and supervision and oversight by healthcare professionals remain essential for safe and effective use. WHAT'S KNOWN: • Children with cerebral palsy (CP) receive lifelong rehabilitation. Exercise plays a fundamental role in maintaining functional independence and quality of life, as well as physical development. The use of AI-based tools such as ChatGPT-4 for health information is increasingly widespread; however, uncertainties regarding the reliability, quality,and readability of these tools remain. WHAT IS NEW: • Our study shows that ChatGPT-4''s responses to questions about treatment and rehabilitation strategies for children with CP have moderate to good reliability and quality, but low readability.

References

1. 1.Zorlular, R., & Zorlular, A. (2026). Evaluation of the quality, reliability, and readability of ChatGPT-4 responses related to the treatment and rehabilitation of children with cerebral palsy. European Journal of Pediatrics. https://doi.org/10.1007/s00590-025-04198-4