Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 3 appraisals
JMIR medical education
Evaluating the Pediatric Behavior Guidance of Students Based on Actual Clinical Transcripts Scored by Faculty and Large Language Models: Pilot Comparative Study
BACKGROUND: Personalized feedback improves the clinical pediatric behavior guidance performance of students but is prohibitively time-consuming to provide. Large language models (LLMs) can automate the process of evaluating clinical sessions but are limited to text-only input and consistency issues. OBJECTIVE: This study compared the use of text-only transcripts against the use of video recordings for evaluating the clinical behavior guidance performance of dental students. Additionally, the consistency and accuracy of LLMs in evaluating the transcripts were compared against a human assessor. METHODS: This study was conducted by using 40 video-recorded clinical encounters involving final-year dental students who were managing patients aged between 4 and 12 years at the Faculty of Dentistry, National University of Singapore. The videos were scored by using a previously validated pediatric behavior guidance scale. Clinical encounters were transcribed verbatim and scored by a study member using a modified version of the scale (nonverbal components removed). The time taken to rate the transcripts was recorded. Video scores were compared with transcript scores. Both the free-to-use version and the paid version of the ChatGPT LLM were also used to score the transcripts; consistency was evaluated and compared against the human assessor. RESULTS: The average time taken to rate the transcripts (mean 12, range 3-25 min) was significantly (P<.001) lower than the average video length (mean 73, range 37-120 min). Comparing transcript scores with video scores resulted in a consistency intraclass correlation coefficient of 0.830 (95% CI 0.679-0.910; P<.001), demonstrating good reliability. Comparing transcript scores with the free-to-use LLM's and paid LLM's scores yielded an absolute agreement intraclass correlation coefficient of 0.729 (95% CI 0.475-0.859; P<.001) and 0.670 (95% CI 0.377-0.825; P<.001), respectively, demonstrating moderate agreement. The LLMs were inconsistent, producing variable scores with the same prompt. The free-to-use and paid versions produced the same score for all 3 runs in only 7 (18%) and 4 (10%) of the 40 clinical encounters, respectively. CONCLUSIONS: Using transcripts to evaluate students' clinical behavior guidance was time-saving for faculty, demonstrated good agreement with video-based evaluation, and could improve clinical teaching. Although LLMs can automate the task, improvements are needed to improve their consistency and accuracy.
14 June 2026
Read appraisal →Dental materials journal
Domain shift analysis of deep learning models for tooth detection in pediatric panoramic radiographs
The aim of this study was to externally validate object detection models for comprehensive tooth detection on pediatric panoramic radiographs, quantify the impact of domain shift across institutions and imaging protocols, and compare YOLOv8 and YOLOv10. Two datasets were used: an internal set of 200 images of early mixed dentition without bite blocks, and an external, open-source set of 192 images acquired with bite blocks. Performance was assessed using mean average precision (mAP), per‑class AP, precision‑recall curves, and confusion matrix. Overall performance was high but domain dependent. mAP was 0.958/0.941 (YOLOv8/v10) in Experiment 1; 0.910/0.906 in Experiment 2; and 0.906/0.901 in Experiment 3. Error analysis revealed domain‑specific failure modes: bite blocks were occasionally mistaken for the primary central incisor. Deep learning enables accurate and comprehensive detection of teeth on pediatric panoramic radiographs. However, domain shift alters error patterns and impairs the detection of rare yet clinically important classes.
25 May 2026
Read appraisal →Calcified tissue international
Diagnostic Indices for Epidemiological Assessment of Molar Incisor Hypomineralization: A Systematic Review
Molar incisor hypomineralization (MIH) is a qualitative developmental defect of enamel and one of the most prevalent oral health conditions in childhood. Over the last two decades, increasing attention has been given to its prevalence and etiology; however, marked heterogeneity in diagnostic criteria and indices has limited the comparability of epidemiological findings. This systematic review aimed to identify and describe the diagnostic indices used for the epidemiological assessment of MIH in the general pediatric population. A systematic literature search was conducted in PubMed, Scopus, and Google Scholar to identify epidemiological studies published between January 2001 and December 2024 reporting MIH prevalence in children. Study selection, data extraction, and methodological quality assessment were performed independently by two reviewers in accordance with PRISMA guidelines. The review protocol was registered in PROSPERO (CRD42022345224). A total of 198 articles were included, corresponding to 201 independent epidemiological studies, as three articles reported results from two distinct study populations. The European Academy of Paediatric Dentistry (EAPD) criteria were used in 171 studies (75.7%), representing the most widely adopted diagnostic index. Other indices, including the modified Developmental Defects of Enamel index (mDDE), Mathu-Muju and Wright, Wetzel and Reckel, MIH Severity Scoring System (MIH-SSS), the MIH Treatment Need Index (MIH TNI) criteria, and FDI criteria, were used infrequently. Studies using EAPD criteria accounted for the largest cumulative sample size and showed a descriptively aggregated MIH prevalence of 12.9%, whereas prevalence estimates varied across alternative diagnostic indices. In conclusion, the lack of uniform diagnostic standards and nomenclature hampers global surveillance and limits the comparability of prevalence data. The adoption of MIH-specific diagnostic criteria, supported by adequate examiner calibration and transparent reporting, is essential to improve the quality, consistency, and interpretability of future epidemiological research.
11 May 2026
Read appraisal →