Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 6 appraisals
JMIR medical informatics
Machine Learning for Intraoperative Bleeding Prediction in Patients Undergoing Surgery: Scoping Review
BACKGROUND: Intraoperative bleeding is a critical event that impacts surgical safety and patient outcomes. Machine learning (ML) has demonstrated potential in prediction tasks, yet its methodological rigor and clinical translation face challenges. OBJECTIVE: This scoping review aims to systematically synthesize the current state of development, performance, and validation of ML models for predicting intraoperative bleeding, and to identify key barriers to their clinical implementation. METHODS: Following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, we systematically searched 7 databases (PubMed, Web of Science, Embase, CINAHL, CNKI [China National Knowledge Infrastructure], Wanfang, and VIP [China Science and Technology Journal Database]) from their inception to April 2025. Moreover, 2 reviewers (SY and PZ) independently screened studies, extracted data using the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS), and assessed the risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST). A narrative synthesis was used for data analysis. RESULTS: Out of 2651 screened records, 23 studies were included (sample sizes ranging from 48 to 48,543). Tree-based ensemble models (eg, random forests and extreme gradient boosting) were the most frequently used (16/23, 70%), followed by logistic regression (13/23, 57%), and deep learning (11/23, 48%). Model discrimination varied widely (mean area under the curve [AUC] 0.82, SD 0.08, range 0.63-0.93). Integration of multimodal data (electronic health records+imaging) was associated with higher performance. However, model validation was often inadequate; only 6 studies (6/23, 26%) performed external validation, and performance often declined (eg, AUC decreased from 0.85 to 0.63 in 1 study). Reporting exhibited selective bias; AUC was commonly reported (19/23, 83%), whereas key classification metrics, such as calibration (10/23, 43%) and precision (4/23, 17%), were often omitted. PROBAST assessment indicated a high risk of bias in all included studies (23/23, 100%). CONCLUSIONS: While ML models demonstrate technical promise for predicting intraoperative bleeding, our PROBAST assessment revealed a universally high risk of bias across all included studies. This fundamental methodological limitation, coupled with a severe lack of external validation and poor transparency in reporting, severely constrains the current clinical reliability of these models. Future research must prioritize prospective multicenter validation, adherence to Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines, and enhanced model interpretability to bridge the gap toward clinical utility.
13 June 2026
Read appraisal →JMIR research protocols
An AI-Enabled Predictive Harm Response Management Algorithmic Tool to Reduce Adverse Events in Health Care Settings (PreHaRM): Protocol for a Three-Phase Model Development and Pilot-Testing Study
BACKGROUND: Current data analysis and coordination methods do not effectively support nurses and midwives in risk reduction, as retrospective reporting does not allow real-time insights and precludes proactive, preventive care. Analysis of administrative data within Australia's health care sector to predict risk may help address this shortcoming. Predictive analytics can transform these data into meaningful insights, identifying harm risk profiles that benefit the performance of Australian and international clinical programs. Importantly, these tools may support nurses and midwives in preventing adverse events and predicting high-risk situations. Researchers, in collaboration with local health network staff, will develop a proof-of-concept predictive risk algorithm. The "predictive harm response management algorithmic tool to reduce adverse events in healthcare settings" program (project DHCRC-0156) will provide real-time insights via an interactive dashboard, enabling nurses, midwives, and health administration to assess risks and optimize resources in health care settings. This protocol details the algorithm development activities for subproject 1a, predictive risk model development, which aims to develop and pilot-test a predictive harm algorithm across 2 South Australian local health networks. OBJECTIVE: This study aims to identify the clinical harm outcome of interest and relevant data sources per site and build a suitable data solution to model predictors of harm risk and identify actionable clinical, workforce, and environmental factors that affect the harm outcome of interest. METHODS: This study design includes three phases: (1) model generation, (2) model evaluation, and (3) prototype development. Data linkage by SA-NT DataLink can only proceed following approval from each of the following: the South Australia Department for Health and Wellbeing Human Research Ethics Committee, the University of South Australia Human Research Ethics Committee, and hospital governance committees. The clinical dataset will be split into a training set, a validation set, and a test set. Exploratory data analysis will be undertaken to ascertain features and classify outcomes from the raw dataset. Derived features will be computed, feature correlations will be estimated, and initial feature selection will be performed. Iterative model development will occur over 3 stages, and a dashboard to display these results will be developed. RESULTS: The study commenced on July 19, 2021, and will conclude on December 31, 2025. Finalized results are expected in December 2025. CONCLUSIONS: This research will conclude with the development of the algorithm for transferability to health care environments. Research activities will be detailed in publicly available reports and manuscripts prepared for peer-reviewed journals that will be drafted in accordance with existing and appropriate checklists. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/75474.
29 May 2026
Read appraisal →PloS one
Enhancing aviation safety: An 80-year data-driven model for classification of aviation incident and accident
The aviation system is safety-critical by nature, and any occurrence of an incident or accident can lead to the loss of human life and significant operational disruptions. The International Civil Aviation Organization (ICAO) emphasizes that every flight must take off and land safely-a goal achieved over 126,000 times daily. Despite major advancements,mishaps and accidents continue to occur, underscoring the need for robust safety management systems. The accurate classification of aviation occurrences (Incident or Accident) reports is essential for safety management, yet manual review is time-consuming and prone to inconsistency. While incident/accident labels are assigned during reporting, automated classification enables rapid triage, detection of potential mislabeling, and support for severity assessment in high-volume aviation safety operations. To address this,we developed and compared three machine learning classifiers-Multinomial Naive Bayes, Random Forest, and Support Vector Machine-using TF-IDF vectorization on an 80-year dataset of 53,770 aviation occurrence summaries obtained from the Transportation Safety Board of Canada. A two-stage evaluation strategy was employed, consisting of an initial 80/20 train-test split to create an independent test set, followed by 5-fold cross-validation applied exclusively to the training data to ensure robustness and prevent optimistic bias. The Support Vector Machine (SVM) classifier achieved the highest classification performance, attaining an accuracy of 98.06% during 5-fold cross-validation, with consistent results across folds, demonstrating its effectiveness in managing high-dimensional textual data and dataset complexity. The proposed framework provides a robust foundation for automated aviation safety report processing, offering practical value for (1) early triage of safety reports, (2) identification of potentially mislabeled cases requiring expert review, and (3) integration into downstream severity assessment pipelines. This work advances beyond prior classification studies by establishing a benchmark on the largest historical aviation safety dataset while delivering a deployable and operationally relevant framework for real-world safety management applications. The findings offer valuable insights for regulatory authorities and airline operators, contributing to enhanced safety oversight, improved response strategies, and safer aviation operations.
23 May 2026
Read appraisal →BMJ open
Prediction of inpatient falls and key predictors using machine learning applied to electronic health records: a retrospective cohort study in a tertiary hospital in Türkiye
OBJECTIVE: Traditional fall risk tools are often inaccurate and burdensome. This study aims to develop a predictive model for inpatient falls and identify the most influential variables using machine learning applied to electronic health record data. DESIGN: A retrospective cohort study. SETTING: A large tertiary university hospital in Türkiye. PARTICIPANTS: Adult patients (≥18 years) hospitalised in a university hospital between January 2017 and June 2023. PRIMARY OUTCOME MEASURES: Occurrence of inpatient falls recorded in incident reporting systems. RESULTS: A total of 518 fallers were identified and compared with 3121 non-fallers. Fallers were significantly older (median 68.5 vs 64 years, p<0.001), had longer hospital stays (16 vs 12 days, p<0.001) and higher comorbidity burden (p<0.001). They were also more likely to receive high fall-risk medications (86% vs 76%, p<0.001) and had a higher prevalence of mental disorders (26.3% vs 17.2%, p<0.001). The random forest quantile classifier demonstrated the highest discrimination (area under the curve (AUC) 0.821, 95% CI 0.781 to 0.861), with balanced sensitivity (0.799, 95% CI 0.734 to 0.867) and specificity (0.722, 95% CI 0.691 to 0.755). Pairwise comparisons using DeLong's test showed significantly higher AUC compared with several models (adjusted p=0.001), while differences with eXtreme Gradient Boosting were not significant (p=0.240). Key predictors included comorbidity burden, age, number of fall-risk-increasing medications and laboratory variables such as eosinophil and basophil counts, blood urea nitrogen, sodium and red blood cell count. CONCLUSION: Machine learning models using electronic health data can predict inpatient falls and reveal key risk factors. The random forest quantile classifier offers a promising approach for improving fall risk prediction in imbalanced clinical datasets.
15 May 2026
Read appraisal →PloS one
Artificial intelligence for monitoring hand hygiene compliance in healthcare settings: A scoping review
BACKGROUND: Hand hygiene is a fundamental measure for preventing healthcare-associated infections, yet traditional monitoring methods are significantly limited by the Hawthorne effect, high resource demands, and an inability to assess procedural quality. Artificial intelligence (AI) technology has emerged as a transformative, automated, and objective approach to address these long-standing challenges. OBJECTIVE: This scoping review sought to systematically map the existing evidence, technical pathways, performance metrics, and implementation challenges of AI for monitoring hand hygiene compliance in healthcare settings. METHODS: Following the Joanna Briggs Institute (JBI) methodological framework and PRISMA-ScR guidelines, we searched five major databases (PubMed, Scopus, Embase, Web of Science, and IEEE Xplore) for articles published between January 2000 and September 2025, supplemented by grey literature searching and backward citation tracking. Two reviewers independently screened records, assessed full-text reports for eligibility, and extracted data, which were synthesized using descriptive statistics and thematic analysis. RESULTS: Of 800 records identified through database and supplementary searches, 45 studies (2007-2025) were included. The primary technical pathways identified were computer vision (53.3%), wearable sensors (24.4%), Internet of Things-integrated systems (13.3%), and radar/radio frequency-based systems (8.9%). While computer vision achieved high accuracy (95%) in setting-specific ICU models, performance dropped to 56% in generalizable models. Wearable systems demonstrated portability but showed 5%-10% lower specificity than vision-based approaches. Most evidence is derived from small-scale technical validations, with a significant lack of formal fairness analysis and evaluation of clinical workflows or cost-effectiveness. CONCLUSION: AI-based hand hygiene monitoring shows promise for supporting more objective and scalable hand hygiene surveillance in healthcare settings. However, the field remains at a largely pre-translational stage. Future research should shift from technical feasibility toward implementation science, focusing on establishing standardized motion databases, evaluating ethical governance (e.g., privacy and automation bias), and conducting pragmatic trials to demonstrate sustained clinical benefit and organizational sustainability.
24 Apr 2026
Read appraisal →Clin Obstet Gynecol
Artificial Intelligence in Obstetrics: Current Applications, Opportunities, and Clinical Implementation Challenges
Artificial intelligence is transforming obstetric practice through applications in diagnostic imaging, risk prediction, and clinical decision-making. Deep learning algorithms have achieved diagnostic accuracy comparable to that of experienced clinicians. However, gaps persist between algorithmic capability and clinical implementation. Critical challenges include limited external validation and algorithmic bias. This review examines current AI applications in obstetrics across multiple clinical domains: automated fetal biometry, structural anomaly detection, prediction of pregnancy complications, and intrapartum fetal surveillance. It highlights persistent technical, ethical, and implementation barriers. Key recommendations include multicenter validation across diverse populations, explainable AI approaches, and creating strong regulatory frameworks.
1 Mar 2026
Read appraisal →