Accuracy of Large Language Models in Answering Dental Examination Questions: A Systematic Review and Meta-Analysis

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract; ChatGPT-4 significantly outperformed earlier versions and some competitor models on direct comparison

• Effect Size: Pooled accuracy: 63.7%; ChatGPT-4 subgroup: ~73%; Microsoft Copilot subgroup: ~75%

• Primary Outcome: Pooled accuracy of LLMs in answering dental examination questions across 39 included studies

• Nnt Or Sensitivity: Not applicable as a performance accuracy metric; no NNT calculable. Passing threshold context: most dental board examinations require 60–70% accuracy, placing overall LLM performance at the lower margin of passing thresholds and GPT-4/Copilot above typical passing benchmarks

• Confidence Interval: Overall: 95% CI 60.3%–67.1%

Clinical Application

LLMs such as ChatGPT-4 and Microsoft Copilot are freely or affordably accessible to dental students and practitioners globally. Integration into examination preparation workflows is technically feasible. However, the high heterogeneity of performance across question types and dental subdisciplines means that reliability cannot be assumed across all use cases. Structured prompting and retrieval-augmented generation approaches are identified as warranting further investigation. In Australia, dental graduates must pass the Australian Dental Council (ADC) written examination for overseas-trained dentists, and domestic graduates complete university-based assessments aligned with ADC competency standards. The included studies do not appear to specifically evaluate ADC examination formats. The TGA does not currently regulate LLMs as medical devices for clinical decision support in dentistry. RACGP and ADC guidelines do not yet formally address LLM use in dental education or examination preparation. Australian dental educators should treat these findings as preliminary and context-dependent. The PBS does not apply to LLM tools. Dental schools considering LLM integration into curricula should note that a 63.7% pooled accuracy — with I² of 91.5% — does not support uncritical adoption, but does not preclude supervised, educationally scaffolded use. Dental students, dental educators, and dental examination bodies considering LLM integration into examination preparation, self-directed learning, or formative assessment tools. Not applicable to clinical patient care decision-making based on current evidence.

Abstract

INTRODUCTION: Large language models (LLMs), including OpenAI's GPT family accessed via interfaces such as ChatGPT and Microsoft Copilot, as well as non-GPT systems such as Google Gemini, are increasingly applied in healthcare and dental education. However, the accuracy of these systems in specialized tasks such as answering dental examination questions remains unclear. METHODS: This systematic review and meta-analysis evaluated LLM performance in answering dental questions. Databases searched were PubMed, Embase, Scopus, and Web of Science. Data on question type and number, LLM versions, and accuracy rates were extracted. Pooled accuracy was estimated using a random-effects model; heterogeneity and publication bias were assessed. RESULTS: A total of 39 studies were included, with ChatGPT-4 being the most frequently evaluated model. The pooled accuracy for LLMs was 63.7% (95% CI: 60.3%-67.1%), with high heterogeneity (I² = 91.5%). Subgroup analysis revealed ChatGPT-4 and Copilot (a GPT-based interface) achieved the highest pooled accuracies (∼73% and ∼75%, respectively). Direct comparisons confirmed ChatGPT-4 significantly outperformed earlier versions and some competitor models. Sensitivity analyses supported the robustness of findings. CONCLUSION: LLMs demonstrate moderate accuracy in answering dental examination questions and are currently insufficient for autonomous clinical decision-making. When their limitations are explicitly recognized, however, these systems may serve as valuable adjuncts in dental education and examination preparation. Methodological strategies such as structured prompting and retrieval-augmented approaches warrant further investigation but were not the primary focus of the present analysis.

References

1. 1.Dashti, M., Khosraviani, F., Meyari, A., Amirzade-Iranaq, M. H., Chaurasia, A., Hefzi, D., Ghadimi, N., Tichy, A., Khurshid, Z., & Schwendicke, F. (2026). Accuracy of large language models in answering dental examination questions: A systematic review and meta-analysis. International Dental Journal. Advance online publication. https://doi.org/10.1002/jrsm.1564