Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 4 appraisals
PloS one
Developing a natural language processing system using transformer-based models for adverse drug event detection in electronic health records
OBJECTIVE: To develop a transformer-based natural language processing (NLP) system for detecting adverse drug events (ADEs) from clinical notes in electronic health records (EHRs). MATERIALS AND METHODS: We fine-tuned BERT Short-Formers and Clinical-Longformer using the processed dataset from the 2018 National NLP Clinical Challenges (n2c2) shared task Track 2. Two data processing methods, window-based and split-based approaches, were compared to identify the optimal processing method. Model generalizability was evaluated on a dataset extracted from Vanderbilt University Medical Center (VUMC) EHRs. RESULTS: On the n2c2 dataset, the best 5-fold cross-validation AUPRC, micro F1, and macro F1 scores were 0.840 (Clinical-Longformer, 4-chunk split), 0.965 (BioBERT, 15-word window), and 0.852 (Clinical-Longformer, 10-chunk split). On the VUMC dataset, the best AUPRC, micro F1, and macro F1 scores were 0.536 (Clinical-Longformer, 6-chunk split), 0.966 (BERT-base-uncased, 6-chunk split), and 0.762 (Clinical-Longformer, 4-chunk split). DISCUSSION: Transformer-based models demonstrated strong performance for ADE detection, with split-based processing generally outperforming window-based methods. Clinical-Longformer combined with a practical split-based approach showed promise for real-world implementation. Beyond token limits, chunk size substantially influenced model performance, even when text length remained within limits. CONCLUSION: Our findings provide practical guidance for developing transformer-based ADE detection systems from clinical notes. The selection of both text preprocessing strategies and model architectures should be guided by note characteristics and practical considerations such as annotation burden.
2 July 2026
Read appraisal →Journal of medical Internet research
A Futures Framework for Clinical AI Governance: Anticipating Emerging Risks, Shifting Roles, and Regulatory Challenges
This viewpoint develops the futures framework for clinical artificial intelligence governance (FF-CAIG), a conceptual and anticipatory framework for organizing emerging governance challenges in clinical artificial intelligence (AI). Although life cycle-oriented oversight is increasingly reflected in clinical AI regulation and institutional governance, existing approaches remain more developed for near-term validation, current-state assurance, and retrospective risk detection than for longer-horizon sociotechnical change. This gap is increasingly relevant, as AI systems become more complex, adaptive, and autonomous, and as they become more deeply embedded in care relationships and accountability structures. FF-CAIG is grounded in 3 futures methodologies: the 3 horizons model, scenario planning, and causal layered analysis. It is operationalized through an emerging clinical AI risk taxonomy that links these methods to governance domains. Its practical outputs include horizon classification, risk-domain mapping, scenario stress-testing findings, accountability-chain mapping, and horizon-scaled minimum governance actions for deployment or continued use. Applied across near-term, transitional, and longer-term horizons, the framework proposes cross-horizon priorities, including stronger predeployment equity evaluation, clearer life cycle accountability, clinician AI oversight competencies, and safeguards for increasingly autonomous or AI-mediated care systems. We illustrate FF-CAIG through 3 representative clinical AI deployment patterns and discuss its limitations, including differential compliance burdens, risks of overdocumentation, variable feasibility across jurisdictions, and the need for empirical validation. FF-CAIG is intended not as a prescriptive policy instrument or validated assessment tool, but as a structured analytic approach for regulators, health system leaders, developers, and researchers seeking prospective and systems-oriented approaches to clinical AI governance.
30 June 2026
Read appraisal →The Cochrane database of systematic reviews
Prognostic models for predicting intensive care unit admission or mortality in critically ill adults not yet been admitted to the intensive care unit
OBJECTIVES: This is a protocol for a Cochrane Review (prognosis). The objectives are as follows: To determine whether prognostic models can be used for predicting the occurrence of ICU admission and mortality within the index hospital admission, and post-discharge survival at reported follow-up time points, up to one year, in critically ill adults not yet admitted to the ICU. The objective in PICOTS format is as follows. POPULATION: critically ill adults not yet admitted to the ICU Index prognostic model: available prognostic models with or without external validation Comparator: not applicable Outcomes: ICU admission, ICU and hospital mortality, and post-discharge survival Timing: for ICU admission or transfer, ICU mortality, and hospital mortality outcomes, the timing will be in hospital. For post-discharge survival, the timing will be up to 28 days, 1 to 3 months, 6 months, and 1 year after hospital discharge. SETTING: acute hospital wards and emergency care departments.
30 June 2026
Read appraisal →PloS one
Hybrid deep learning for mental workload classification using EEG with enhanced preprocessing and interpretability
Mental workload classification is critical in safety-sensitive fields such as healthcare and aviation. However, electroencephalography-based approaches still face challenges with generalizability, noise robustness, and interpretability. In this study, we propose an integrated hybrid deep learning framework to address these limitations and enable robust, interpretable electroencephalography-based mental workload classification. The proposed approach uses a Variational Autoencoder to enhance noise reduction and feature extraction from band-wise topographical videos, a Convolutional Block Attention Module to adaptively focus on important spatial-channel Electroencephalogram features, and a Bidirectional Long Short-Term Memory network to capture complex temporal dependencies under leave-one-subject-out cross-validation. We conducted ablation studies to identify each architecture component's contribution and sensitivity analyses to determine the optimal parameters. The model achieved the highest overall accuracy among the baselines and reached an average accuracy of 83.9% across subjects for classifying four mental workload levels. Ablation studies confirmed the added value of all three architecture components for improving performance. Sensitivity analyses identified optimal parameters, including a 10-second window length that balances temporal context and specificity. For better interpretability and neurophysiological insight, we used Gradient-weighted Class Activation Mapping to visualize key frontal-parietal brain regions and frequency bands associated with workload dynamics. Future research could explore adaptive windowing strategies, multimodal data integration, cross-dataset benchmarking, and evaluation against transformer-based and graph neural network architectures under consistent subject-independent evaluation settings to further enhance model generalizability.
28 June 2026
Read appraisal →