Large Language Models for Endodontic Symptom Assessment and Treatment Planning Using Image-Free Clinical Records: Comparative Evaluation Study

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract

• Effect Size: ChatGPT-4.0 achieved diagnostic accuracy of 0.65 for pulpal disease and 0.57 for periapical disease, comparable to AGD and endodontic residents. AGD specialists achieved the highest human accuracy (pulpal: 0.70; periapical: 0.65). ChatGPT's best mean concordance score was 0.98/2.0 (Korean-doctor prompt), which did not reach the 'partially correct' threshold of 1.0. Clova X performed worst (mean 0.23/2.0 on English-patient prompts).

• Primary Outcome: Diagnostic concordance and accuracy for pulpal and periapical status assessment using text-based clinical records without radiographic images

• Nnt Or Sensitivity: Sensitivity and specificity reported qualitatively: AGD specialists showed higher sensitivity but lower specificity than other groups. Quantitative sensitivity/specificity values not provided in abstract. No NNT applicable (diagnostic study).

• Confidence Interval: ChatGPT pulpal accuracy: 95% CI 0.55–0.74; periapical accuracy: 95% CI 0.47–0.69

Clinical Application

LLMs are freely or commercially accessible (ChatGPT-4.0 via OpenAI subscription), requiring no additional infrastructure. However, clinical deployment requires robust governance frameworks, clinician oversight, and awareness of hallucination risk. The image-free constraint studied here does not reflect standard endodontic workup, limiting direct clinical translation. In Australia, endodontic treatment is provided by both general dentists and specialist endodontists. Root canal therapy is not subsidised under the Medicare Benefits Schedule for adults, though limited coverage exists under the Child Dental Benefits Schedule (CDBS). The Dental Board of Australia and AHPRA require that clinical decisions remain the responsibility of the registered practitioner — LLM outputs cannot substitute for clinical judgement or satisfy medicolegal standards of care. The TGA does not currently regulate general-purpose LLMs as medical devices unless specifically marketed for diagnostic purposes, though this regulatory landscape is evolving. RACGP and the Australian Dental Association (ADA) have not yet issued formal guidance on LLM use in clinical dentistry. Australian general dental practitioners working in rural and remote settings, where specialist referral is limited, may represent a population where LLM decision support tools could offer the greatest marginal benefit, but this requires prospective validation in Australian clinical contexts. Dental practitioners and trainees considering LLM-assisted clinical decision support for endodontic diagnosis and treatment planning, particularly in settings where specialist expertise is limited or as an adjunct educational tool for dental students and junior residents

Abstract

BACKGROUND: Accurate assessment of pulpal status is essential for achieving successful endodontic outcomes. However, direct evaluation remains inherently challenging because the pulp is surrounded by calcified tissue, necessitating reliance on clinical and radiographic examinations for diagnostic and prognostic decision-making. These procedures demand substantial clinical expertise and time, and less-experienced clinicians often face challenges that may lead to errors in diagnosis and treatment planning. Recent advancements in large language models (LLMs) offer promising opportunities to enhance clinical reasoning by facilitating the integration of evidence and supporting methodical diagnostic decision-making. OBJECTIVE: This study aimed to evaluate the clinical applicability of LLMs by comparing their text-based clinical screening performance and the clinical validity of their treatment plan responses with those of human evaluators. METHODS: Between January 2011 and December 2022, 100 clinical cases involving primary endodontic disease were randomly selected from the clinical records of outpatients who visited the Department of Conservative Dentistry or Advanced General Dentistry (AGD) at Yonsei University Dental Hospital. Four prompt types, combining 2 variables (language and role), were used as input for 4 LLMs. Both LLMs and human evaluators (AGD specialists, AGD residents, endodontic residents, and senior dental students) assessed the cases using text-based clinical records. Radiographic images were not directly provided. Screening performance was evaluated using a 0-to-2-point concordance scale, and treatment plan validity and relevance were assessed using a 5-point Likert scale. RESULTS: Among the 4 LLMs evaluated, ChatGPT achieved the highest mean concordance score on Korean-doctor prompts (mean 0.98, SD 0.82). However, this score did not reach the partially correct criterion of 1 on the 0 to 2-point scale. Clova X recorded the lowest mean score on English-patient prompts (mean 0.23, SD 0.63). Across both diagnostic categories, AGD specialists demonstrated the highest diagnostic accuracy (pulpal: 0.70; periapical: 0.65), with higher sensitivity but lower specificity than those exhibited by the other groups. ChatGPT also showed favorable performance among the LLMs, with accuracies of 0.65 (95% CI 0.55-0.74) for pulpal disease and 0.57 (95% CI 0.47-0.69) for periapical disease, which were comparable to those of AGD and endodontic residents. CONCLUSIONS: Under image-free clinical record review conditions, ChatGPT 4.0 showed relatively higher and more consistent performance in symptom screening and treatment planning compared to the other LLMs evaluated. However, its highest mean score of 0.98 (SD 0.82) did not reach the partially correct criterion of 1 on the 0 to 2-point scale. Hallucinations generated by LLMs and experience-dependent interpretation biases among human evaluators remain key challenges that require attention. Therefore, continuous clinical supervision and comprehensive user training are necessary for the safe and effective clinical application.

References

1. 1.Seo, D., Cheong, J., Choi, Y., Shin, Y., & Park, W. (2026). Large language models for endodontic symptom assessment and treatment planning using image-free clinical records: Comparative evaluation study. JMIR Medical Informatics. https://doi.org/10.2196/86145