Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 6 appraisals
Proceedings of the National Academy of Sciences of the United States of America
Large language models accurately identify decision reasons in verbal reports.
Understanding the reasons behind human choices under risk is a central goal of decision scientists, but traditional methods relying on behavioral data are limited by strict invariance assumptions. We introduce a scalable analytical framework using large language models (LLMs) to analyze verbal reports and identify articulated reasons for choice between monetary lotteries. A validated LLM accurately identified predefined decision reasons in participants' free-text reports, aligning with their actual choices in 95% of trials. Our analysis reveals that the reasons behind people's decisions vary systematically and are driven more by the structure of the choice problem than by individual differences. Crucially, reasons identified from verbal reports yield more parsimonious and informative representations of decision processes compared to those inferred from choices alone; furthermore, problem-specific reason profiles achieve out-of-sample prediction accuracy that is competitive with established computational models. This work demonstrates that verbal reports are a rich data source and our analytical framework can unlock their potential, delivering results that challenge the field's foundational invariance assumptions and pave the way for more context-sensitive and interpretable models of human decision making.
1 July 2026
Read appraisal →Studies in health technology and informatics
Exploring a Large Language Model-Based Chatbot Use in Data Analysis: A Case Study of the Problems Related to the Do Not Attempt Resuscitation Order
The interpretative framework was used to explore how healthcare professionals (HCPs), artificial intelligence (AI), and researchers construct and negotiate meaning, trust, and authority in critical, highly ethical medical contexts. This case study aimed to explore and test the integration of the Microsoft Copilot® large language model (LLM) chatbot into ethically sensitive clinical decision-making (DM) to identify the problems HCPs face in do-not-attempt-resuscitation (DNAR) order-making and key areas for development. A sample of 100 HCPs' views was analyzed using qualitative content analysis. AI identified 15 thematic categories emerging from responses that described issues related to DNAR protocols and DM. The frequency of opinions in categories ranged from 4 to 16 (Copilot®) and from 2 to 16 (researchers). Based on a three-level frequency scale, AI's interpretations were especially evident in categories describing the roles of patients and families as well as practical issues related to DNAR orders. In contrast, the researchers focused on the importance of communication and documentation.
24 May 2026
Read appraisal →PloS one
Systematic evaluation of the DeepSeek large language model for clinical diagnostic reasoning
BACKGROUND: Artificial intelligence (AI) is undergoing an era of transformative advancement, particularly through the emergence of Transformer-based large language models (LLMs). While these systems demonstrate strong reasoning and generalization capabilities, their clinical applicability, particularly in emergency and critical care decision-making, remains underexplored.. In time-sensitive settings, diagnostic reasoning must align rigorously with evidence-based standards and ensure the relevance of timing to clinical decisions. OBJECTIVE: This study aims to provide a preliminary evaluation of the decision-support performance of the DeepSeek model in acute medical scenarios. We systematically evaluate its diagnostic reasoning, temporal consistency of recommendations, and adherence to evidence-based critical care protocols using standardized case-based assessments. METHODS: Twenty-nine representative clinical cases were extracted from the Merck Manual of Diagnosis and Therapy, a widely used medical reference providing standardized case descriptions. The model's outputs were evaluated across four decision-making dimensions: differential diagnosis, diagnostic testing, final diagnosis, and management planning. Human raters scored each response for accuracy, and multivariable linear regression was applied to assess associations between performance and case parameters (age, gender, and Rapid Emergency Medicine Score [REMS]). RESULTS: DeepSeek achieved an overall mean accuracy of 82.9% (95% CI: 80.2-85.6%) across all cases. Accuracy peaked in final diagnosis (97.7%), but declined in differential diagnosis (73.0%). Model performance showed no significant variation across demographic or severity strata. CONCLUSIONS: DeepSeek shows promising performance in structured case-based diagnostic tasks, particularly in confirmatory diagnostic reasoning. However, its early-stage reasoning and handling of ambiguous cases require enhancement. Future studies using larger and more diverse clinical datasets are needed to further evaluate the model's robustness and potential clinical applicability.
13 May 2026
Read appraisal →Blood advances
How to determine the optimal duration of anticoagulation for VTE: an evidence-based decision-analytical approach
When advising patients on the optimal duration of anticoagulation for venous thromboembolism (VTE), current management often relies on the intuition of treating physicians and guideline panels, an approach that remains opaque to outside observers. We present a decision-analytical framework adopted in our practice. This approach formalizes the intuition of physicians and guideline panels, providing an explicit and transparent integration of the best available evidence on benefits and harms. It also incorporates a systematic method for eliciting patient values and preferences to determine the optimal duration of anticoagulation. To demonstrate the practical utility of the decision-analytical approach, we apply it to 3 clinical scenarios: unprovoked VTE, VTE related to long-haul travel, and portal vein thrombosis in cirrhosis. By transparently synthesizing evidence and patient values, this method supports individualized, evidence-based, patient-centered recommendations for the duration of anticoagulation, and it is applicable to solving similar clinical problems in hematology and medicine.
14 Apr 2026
Read appraisal →Lancet (London, England)
Deferral of percutaneous coronary intervention in patients undergoing transcatheter aortic valve implantation (PRO-TAVI): an investigator-initiated, multicentre, open-label, non-inferiority, randomised controlled trial
BACKGROUND: Coronary artery disease is common in patients undergoing transcatheter aortic valve implantation (TAVI). We aimed to assess whether deferral of percutaneous coronary intervention (PCI) is non-inferior to routine PCI before TAVI in patients with coronary artery disease. METHODS: In this investigator-initiated, open-label, randomised controlled trial, done at 12 hospitals in the Netherlands, TAVI patients with coronary artery disease were randomly assigned in a 1:1 ratio to deferral of PCI or PCI before TAVI. Randomisation was done by use of a web-based system with random block sizes of 2 and 4, and stratification by presence of coronary artery disease involving proximal left anterior descending artery. The primary endpoint was a composite of all-cause mortality, myocardial infarction, stroke, and major bleeding at 1 year. Non-inferiority testing was done in the intention-to-treat population against the prespecified margin of 11 percentage points. The study is registered with ClinicalTrials.gov (NCT05078619) and long-term follow-up is ongoing. FINDINGS: Between Oct 7, 2021, and Nov 19, 2024, 466 patients were enrolled: 233 were assigned to deferral of PCI and 233 to PCI before TAVI. Median age was 81 years (IQR 78-84), and 166 (36%) of 466 patients were female. The primary endpoint occurred in 56 (24%) of 233 patients in the deferral group as compared with 60 (26%) of 233 patients in the PCI group (rate difference -1·7% [95% CI -9·5 to 6·2]; hazard ratio 0·89 [95% CI 0·62-1·28]; p=0·0008 for non-inferiority; p=0·68 for superiority). INTERPRETATION: In patients with coronary artery disease undergoing TAVI, deferral of PCI was non-inferior to PCI before TAVI for the 1-year composite of all-cause mortality, myocardial infarction, stroke, and major bleeding. These findings suggest that an initial conservative strategy can be appropriate in selected patients, although patient-tailored treatment decisions remain essential. FUNDING: ZonMw.
14 Apr 2026
Read appraisal →Heart (British Cardiac Society)
Parsimonious machine learning model to predict 1-year mortality and procedural futility after transcatheter aortic valve replacement
BACKGROUND: Current risk scores inadequately predict long-term mortality after transcatheter aortic valve replacement (TAVR), limiting their ability to guide decisions around procedural futility. We aimed to develop and externally validate a machine learning (ML) model using only preprocedural variables to predict 1-year all-cause mortality. METHODS: An ML model was trained on a retrospective cohort of 1025 TAVR patients using 52 clinical and echocardiographic variables. Feature selection and model tuning were performed via a multiobjective evolutionary algorithm to optimise predictive performance and model simplicity. The final model used 13 preprocedural variables and was externally validated in an independent cohort of 270 patients. Performance was compared with European System for Cardiac Operative Risk Evaluation II (EuroSCORE II), FRANCE-2 and TAVI2-SCORE using the area under the curve (AUC), calibration and net reclassification improvement (NRI). RESULTS: The ML model demonstrated excellent discrimination, with AUCs of 0.81 in the discovery cohort and 0.84 in the external validation cohort. This exceeded the performance of EuroSCORE II (AUC: 0.61 and 0.70), FRANCE-2 (0.52 and 0.58) and TAVI2-SCORE (0.56 and 0.64). Calibration plots showed strong agreement between predicted and observed risks. NRI in the test set compared with FRANCE-2 was 0.62 (95% CI: 0.49 to 0.75); compared with TAVI2-SCORE, it was 0.36 (95% CI: 0.14 to 0.61). The final model incorporated age, atrial fibrillation, creatinine, haemoglobin, pulmonary function, frailty markers (Katz Index, poor mobility) and tricuspid regurgitation. Misclassification analysis revealed that most errors were clustered near the decision threshold, with no evidence of systematic bias. Performance was consistent across subgroups and robust to temporal and institutional variation. CONCLUSION: This externally validated ML model, using 13 routinely available variables, significantly outperforms existing risk scores in predicting 1-year mortality post-TAVR. Its simplicity and generalisability support its potential use in real-world clinical decision-making to identify patients at high risk of procedural futility.
12 Apr 2026
Read appraisal →