Evaluation of GPT-5 in Periodontitis Staging and Grading: Retrospective Observational Study

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Staging accuracy 68%, Grading accuracy 77.3%

• Primary Outcome: Diagnostic accuracy for periodontitis staging and grading

• Nnt Or Sensitivity: Cohen's κ: Staging 0.454 (fair agreement), Grading 0.179 (poor agreement)

• Confidence Interval: Staging: 48.4%-82.8%, Grading: 56.6%-89.9%

Clinical Application

Low feasibility for clinical use given poor diagnostic performance, especially for disease grading Not directly applicable to Australian dental practice due to study limitations and lack of validation in local populations Limited applicability due to small sample and artificial case selection

Abstract

BACKGROUND: Periodontitis is a chronic gum disease affecting approximately 42% of adults aged 30 years and older in the United States. Training dental students to accurately diagnose and manage periodontitis is a critical component of dental education and clinical care. Recent advances in large language models offer new opportunities to support both domains, yet their performance in periodontal diagnosis remains largely unexplored, particularly for newer models such as GPT-5. OBJECTIVE: This study conducted an exploratory evaluation of GPT-5's ability to stage and grade periodontitis. METHODS: A total of 25 publicly available clinical cases explicitly reporting periodontitis stage and grade were identified through Google and PubMed searches. Each case description was entered into GPT-5 using a zero-shot prompting approach to assess guideline-based reasoning without exemplar conditioning. The model's predictions were compared with the published reference diagnoses. Performance was measured using accuracy, 95% CI, unweighted Cohen κ, and weighted Cohen κ. RESULTS: Across these cases, GPT-5 showed marked class-dependent performance and a tendency to overestimate disease severity. Grading performance was notably imbalanced, with high recall for grade C but substantially lower discrimination for grade B. GPT-5 achieved a staging accuracy of 68% (95% CI 48.4%-82.8%) and a grading accuracy of 77.3% (95% CI 56.6%-89.9%), with corresponding Cohen κ values of 0.454 (95% CI 11.0%-75.6%) and 0.179 (95% CI -15.8% to 63.8%), respectively. While staging performance showed fair agreement beyond chance, the low κ for grading indicates poor agreement and limited reliability in distinguishing periodontal disease severity. CONCLUSIONS: These findings suggest that although GPT-5 demonstrates potential for guideline-based periodontitis staging and grading, its current diagnostic performance, particularly for periodontitis grading, limits its use in clinical assessment and educational training. Meaningful application in periodontal diagnosis and training will require substantial improvements in reliability and rigorous validation in larger, more diverse, and prospectively collected datasets.

References

1. 1.Amugo, I., Frederickson, K. L., Rajakaruna, H., Xie, H., Gangula, P., Shanker, A., & Wang, Q. (2026). Evaluation of GPT-5 in Periodontitis Staging and Grading: Retrospective Observational Study. JMIR Formative Research. https://doi.org/10.2196/88407