Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 53 appraisals
Simulation in healthcare : journal of the Society for Simulation in Healthcare
Future of Biometric Technology in Healthcare Simulation
Biometric technology shows substantial promise to enhance patient safety and transform provider training. In patient care, biometric technology enables outpatient providers to optimize chronic disease by alerting providers to changes in patient conditions with real-time data. In provider training, biometric technology can inform training environments and curriculum. Certainly, there are hurdles to technology implementation including hardware fragility, cost, and software integration. Despite challenges, biometric technology has been effectively implemented in simulations and real patient settings. Looking forward, future developments focused on data security and privacy, standardizing data collection, and improving system compatibility will facilitate the wider adoption of biometric technology and improve health care quality, especially in resource-limited settings. Collaborative efforts among informatics experts, patient safety advocates, and health care providers are essential to overcome challenges and fully leverage biometric technology's potential in enhancing patient outcomes. The purpose of this commentary is to review current applications, identify key challenges, and offer recommendations to the simulation community for testing biometric technology, use in training programs, and facilitating systems integration to improve outcomes.
2 Aug 2026
Read appraisal →Surgery
Artificial intelligence to revolutionize surgical decision-making is still just around the corner.
Medical artificial intelligence, especially large language models, has engendered both excitement and unease across the medical community, promising improved surgical diagnostic accuracy, perioperative decision-making, and patient safety amidst concerns of considerable bias and its implications on the future of human expertise. The evolution of artificial intelligence clinical decision support originated in the surgical field through early systems like AAPHelp, which leveraged Bayesian reasoning to diagnose acute abdominal pain. Despite the initial promise of artificial intelligence clinical decision support, attempts to develop more robust diagnostic tools largely fell short in the 1980s, with instruments unable to adequately diagnose and manage complex presentations, a lack of transparency in their decision-making processes, and limitations in scope. It was not until the 2010s that clinical decision support began to show renewed potential as a meaningful adjunct in surgical practice, largely driven by advancements in machine learning techniques and wider access to large clinical data sets. New artificial intelligence models, particularly those utilizing large language models, now demonstrate impressive capabilities, from predicting the risk of postoperative complications for individual patients to streamlining clinical documentation and beyond. However, challenges persist regarding transparency, bias, reliability, and integration into clinical workflows. Despite these hurdles, artificial intelligence-based tools like large language models represent an exciting new chapter in surgical decision-making. Built on a long history of clinical decision support systems in surgery, these technologies hold great promise to meaningfully augment surgical practice and improve care for patients.
1 Aug 2026
Read appraisal →PLoS medicine
How to benchmark medical AI agents
Medical artificial intelligence research is shifting from single-task models toward multimodal large language model-based agents for complex clinical workflows, requiring benchmarks that assess clinical reasoning, process safety, and resource stewardship rather than final outputs alone.
12 July 2026
Read appraisal →BMJ open
Pedagogical strategies for enhancing critical thinking in nursing students using large language models: a mixed-methods systematic review protocol.
INTRODUCTION: Large language models (LLMs) are rapidly normalising in nursing education. However, evidence regarding their impact on critical thinking-a cornerstone of safe clinical practice-remains inconsistent. Current syntheses have largely focused on student satisfaction and general utility, leaving the specific instructional mechanisms that cultivate this competency unexplored. This mixed-methods systematic review aims to identify effective pedagogical strategies for fostering critical thinking, providing evidence-based guidance to safeguard clinical competency in an artificial intelligence-integrated workforce. METHODS AND ANALYSIS: This review will follow a convergent segregated mixed-methods design in accordance with the Joanna Briggs Institute (JBI) methodology. A systematic search will be conducted across CINAHL, ERIC, MEDLINE, PsycINFO and Scopus supplemented by grey literature sources (ProQuest Dissertations, MedRxiv). The search will be limited to English-language studies published from January 2023 to June 2026. Eligible studies will include undergraduate nursing students exposed to structured LLM-based pedagogical interventions. Methodological quality will be appraised using design-specific JBI Critical Appraisal Checklists. Outcomes will be classified using a modified Kirkpatrick framework to distinguish objective competence from subjective confidence. Quantitative data will be synthesised narratively following Synthesis Without Meta-analysis guidelines, and qualitative data thematically. Streams will be integrated using a Joint Display Analysis to generate meta-inferences on effective instructional design. Finally, confidence in the cumulative evidence will be assessed using Grading of Recommendations Assessment, Development and Evaluation (GRADE) and GRADE-Confidence in the Evidence from Reviews of Qualitative research. ETHICS AND DISSEMINATION: As a systematic review of existing literature, this study does not require ethical approval. Findings will be disseminated through a peer-reviewed publication, conference presentations and a visual guide for educators.
11 July 2026
Read appraisal →Clinical infectious diseases : an official publication of the Infectious Diseases Society of America
More Than a Message: Death by a 1000 'Chats'.
Secure electronic messaging within the electronic health record (EHR), such as Epic Chat, has transformed communication in hospitals by enabling rapid, asynchronous communication. Yet for Infectious Diseases (ID) consultants, the constant barrage of messages comes at a cost: fragmented attention, disrupted clinical reasoning, and potential patient harm. Messages that are repetitive, trivial, or casually worded can also fuel miscommunication and even medicolegal risk. As this type of messaging becomes increasingly prevalent in clinical practice, we must establish clearer boundaries and a cultural shift that values purposeful communication. Practical strategies include defining what requires chat versus direct phone or in-person conversation, batching responses, protecting focus time, and teaching messaging etiquette. To optimize patient care and protect clinician well-being, we must be intentional about when and how we communicate.
9 July 2026
Read appraisal →Medical science monitor : international medical journal of experimental and clinical research
Advances in Remote Monitoring Technology Applications in Anesthesia: A Narrative Review
The swift progress of digital and sensor technologies is hastening the incorporation of remote monitoring into anesthesiology. While several reviews have explored telemedicine and artificial intelligence in anesthesia, most existing summaries either focus on conceptual outlook or lack systematic comparison of technical platforms and original clinical validation data. The present review provides a comprehensive, clinically oriented synthesis of remote monitoring in anesthesia, with clear focuses on technical principles, perioperative applications, platform comparisons, and evidence-based clinical outcomes. We critically assess clinical advantages and implementation hurdles, covering the full perioperative pathway - preoperative evaluation, intraoperative observation, and postoperative recovery. A head-to-head comparison of fifth generation of cellular network technology (5G), the Internet of Things (IoT), and cloud computing platforms is presented to clarify their infrastructure demands and suitability for real-world clinical scenarios. Notably, this review summarizes available clinical evidence, including evidence from the Trial of Remote Continuous versus Intermittent National Early Warning Score Monitoring after major surgery (TRaCINg) - a feasibility randomized controlled trial whose exploratory findings suggest that continuous remote monitoring may reduce unplanned intensive care unit admissions, shorten hospital stays, and facilitate earlier detection of postoperative complications. We further propose innovative solutions including multimodal data fusion and federated learning-driven predictive analytics to overcome current limitations in data interoperability, security, and clinical effectiveness. Accordingly, this paper synthesizes the latest advances in remote monitoring technology in anesthesia, clarifies its unique value relative to existing reviews, and provides practical guidance for clinical translation and future research.
2 July 2026
Read appraisal →BMJ open
Effectiveness and safety of pharmacist prescribing: a systematic review
OBJECTIVE: To examine the effectiveness and safety of pharmacist prescribing across multiple healthcare settings. DESIGN: A systematic review of quantitative studies. ELIGIBILITY CRITERIA FOR THE SELECTION OF STUDIES: Quantitative studies assessing the effectiveness and safety of pharmacist prescribing compared with non-pharmacist prescribing in any healthcare setting and for any healthcare population. A clear statement of pharmacists' prescriptive authority was required for inclusion in this systematic review. DATA SOURCES: A systematic search was conducted using six electronic databases: Embase (Ovid), MEDLINE (EBSCO), SCiELO, Dimensions AI, Cochrane Library and Epistemonikos. Database searches were conducted from database inception to 29 January 2025. Additional grey literature searches were conducted using Google and DuckDuckGo. Both backward and forward citation chasing were conducted for all included studies. DATA EXTRACTION AND SYNTHESIS: Data were extracted using standardised bespoke data extraction forms. The revised Cochrane Risk of Bias 2 tool for randomised controlled trials and the Risk of Bias in Non-Randomised Studies of Interventions tool were used to assess risk of bias. A narrative synthesis approach was applied following the Synthesis Without Meta-analysis guideline. The Grading of Recommendations Assessment, Development and Evaluation approach was used to assess the level of certainty of the evidence. RESULTS: Of the 39 included studies, 32 studies reported on effectiveness and 20 studies reported on safety across 15 health conditions. Healthcare settings included outpatient (n=14), primary care (n=10), community pharmacy (n=6), inpatient (n=5), emergency department (n=1) and long-term care (n=3). These studies were based in the USA (n=26), Canada (n=5), the UK (n=4), Australia (n=2) and Singapore (n=2). In total, there were 153 outcomes related to safety and effectiveness. For 74 outcomes, no significant difference was reported between pharmacist prescribing and non-pharmacist prescribing, while 46 outcomes were significantly improved with pharmacist prescribing. Four outcomes reported in favour of non-pharmacist prescribing. Inferential statistics were not reported for 29 outcomes, meaning we cannot comment on their statistical significance. The certainty of evidence was low or very low for all outcomes. CONCLUSIONS: The consistency of effectiveness and safety findings across studies, showing either no significant difference (indicating equivalence of care and outcomes) or significant improvement in pharmacist prescribing groups, suggests it is a potential policy option. Future research on implementation, public and patient preferences and cost-effectiveness would provide valuable insights into the potential benefits of pharmacist prescribing at a health system level.
28 June 2026
Read appraisal →JMIR nursing
Nurses' Experiences Using AI in Clinical Practice: Systematic Review
BACKGROUND: Artificial intelligence (AI) tools are increasingly used in clinical settings, yet most syntheses focus on nurses' attitudes or readiness rather than experiences after direct use in practice. OBJECTIVE: This study aimed to synthesize registered nurses' experiences of using AI in clinical practice and to identify perceived benefits, barriers, and implementation implications. METHODS: We conducted a systematic literature review of empirical studies reporting nurses' experiences of AI use in clinical settings. Searches were performed in CINAHL, Embase, MEDLINE, PsycINFO, and PubMed (last search: September 13, 2023). Moreover, 2 reviewers (AJSS and QZ) independently screened titles, abstracts, and full texts and appraised included studies using the Mixed Methods Appraisal Tool. We used thematic synthesis with a primarily deductive framework based on the Technology Acceptance Model 2, with the addition of facilitating conditions from the unified theory of acceptance and use of technology. RESULTS: In total, 20 studies met the inclusion criteria. Perceived usefulness and facilitating conditions were most frequently reported; nurses described AI as supporting decision-making, workflow efficiency, and confidence when implementation included adequate training, interoperability, and technical infrastructure. Ease of use was closely tied to interface design and training. Job relevance and output quality were rated more positively when nurses described AI as aligning with nursing tasks and as producing interpretable, reliable outputs. Common barriers included usability issues, limited integration into workflows and electronic systems, privacy and trust concerns, and inconsistent or poorly contextualized outputs. Across studies, nurses often described adoption as conditional on organizational readiness and meaningful involvement of nurses in design and implementation. CONCLUSIONS: Nurses' accounts suggest that AI may augment clinical work, but the perceived benefits reported in the included literature were contingent on workflow alignment, usable interfaces, training, and supportive infrastructure. These findings suggest potential value in involving nurses in the co-design and iterative refinement of AI tools, grounded in the consistent evidence that facilitating conditions, such as training, interoperability, and organizational readiness, are the primary determinants of whether AI is experienced positively. Prospective evaluation using nursing-relevant outcomes, such as usability, workflow integration, and trust, is needed to move beyond post hoc experiential accounts and establish what implementation conditions reliably produce benefit.
26 June 2026
Read appraisal →BMJ open
Development of phenotype algorithms for the detection of adverse events in electronic health record data: a multicentre study
OBJECTIVES: To develop phenotype algorithms for the detection of adverse events (AEs) or AE-proxies in electronic health records (EHRs), accounting for varying data availability. DESIGN: Multicentre study conducted as part of the Use Case POLAR_MI (POLypharmacy, drug interActions, Risks) of the German Medical Informatics Initiative (MII). SETTING: Germany. PARTICIPANTS: Multidisciplinary teams from 10 German university sites within the MII. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Literature- and consensus-based development and operationalisation of AE algorithms using structured EHR data, including a standardised, multicentre expert review process. Data categories used: International Classification of Diseases, 10th Revision (ICD-10) codes for diagnoses; Anatomical Therapeutic Chemical (ATC) codes and 'Pharmazentralnummern' (PZN; German eight-digit identification code for pharmaceutical products) for medications (used in the treatment of AEs); Logical Observation Identifiers Names and Codes (LOINC) for laboratory values and medical findings; and 'Operationen- und Prozedurenschlüssel' (OPS; German procedure classification) codes for medical and surgical procedures. RESULTS: We developed 82 algorithms for 48 AEs. Algorithms for the same AE varied by data categories or code selections. At the AE level, 31 AEs were covered exclusively by newly developed algorithms, and 17 AEs by at least one modified algorithm.Overall, 52 algorithms were based on a single data category, while 30 required multiple categories. ICD-10 codes were most commonly used (n=65 AE algorithms), followed by LOINC (n=27), ATC codes (n=18), OPS codes (n=11) and PZN (n=2). All phenotype algorithms were semantically modelled and can be executed using the publicly available Terminology- and Ontology-based Phenotyping (TOP) Framework, which supports export in various formats. CONCLUSION: We present a peer-reviewed set of algorithms for a large number of AEs, which can be implemented in structured routine electronic data sources and (pending validation studies) may support pharmacoepidemiologic research. The algorithms will be implementable across all 39 participating sites of the German MII. As a next step, we will empirically validate the algorithms against all information (including free text) contained in EHRs. TRIAL REGISTRATION: Not applicable.
24 June 2026
Read appraisal →Proceedings of the National Academy of Sciences of the United States of America
AI agents are sensitive to nudges.
Large language models (LLMs) are increasingly deployed as autonomous agents that make choices and use tools on behalf of users. Yet, we have limited evidence about how their decisions are shaped by their environment. We adapt a human decision-making task to test leading LLMs under four forms of choice architecture: defaults, suggestions, information highlighting, and "optimal" nudges derived from a resource-rational model of human choice. We treat human behavior as a baseline for predictable sensitivity to such interventions. Across models and prompting strategies, LLMs often depart substantially from this baseline. They sometimes pay excessive costs to acquire information, sometimes ignore available information, and, most crucially, are far more responsive to nudges than humans, such that weak cues that slightly shift human behavior have larger effects on model choices, toward both better and worse payoff outcomes. Chain-of-thought prompting and in-context human data do not reliably stabilize behavior. Recent reasoning-optimized LLMs can, in some configurations, restore more human-level sensitivity to nudges, but do so inconsistently and at substantial computational cost. These results point to an important and largely neglected safety concern: LLM agents can be behaviorally brittle under subtle changes in choice architecture, even in the absence of adversarial settings.
23 June 2026
Read appraisal →European journal of emergency medicine : official journal of the European Society for Emergency Medicine
Association between patient-to-nurse ratios in emergency departments and patient, staff, and organizational outcomes: a systematic review.
Background In many settings, studies have shown that establishing patient-to-nurse ratios has benefits for both patients and staff. Whether this is transposable to emergency department (ED) is unclear. Objective The primary objective was to assess the association between patient-to-nurse ratios in EDs and mortality. Design In this systematic review, a search strategy identified studies published in 2004-2025 in MEDLINE, Cochrane, and CINAHL. Two independent reviewers performed selection and data extraction, followed by a narrative synthesis. Outcomes The primary outcome was in-hospital mortality. Secondary outcomes were other patient outcomes, staff, and organizational outcomes. Results Of 2214 identified references, 17 studies were included and all of which were observational. The primary outcome was assessed in only one cohort study ( n = 73 EDs), where an increase in the number of nurses per patient was associated with a reduction in ED mortality (adjusted odds ratio = 0.68; 95% confidence interval: 0.49; 0.94). An increase in the number of nurses per patient was also associated with a reduction in cardiac arrests (one study: relative risk = 1.33, 95% confidence interval: 1.05, 1.67 for ≥8.5 patients/nurse vs. <8.5) and medication errors (one of two studies), and an improvement in patient experience (two studies) and nurse satisfaction (one study). Mixed results were observed with regard to reducing length of stay (one of two studies), administration time (three of four studies) and the number of patients who left without being seen (three of four studies), while waiting time before diagnosis or treatment improved (three studies). Conclusion Although evidence is limited, several observational studies suggest that a higher nurse-to-patient ratio is associated with better patient, nurse, and ED organizational outcomes.
23 June 2026
Read appraisal →Current opinion in anaesthesiology
Traditional learning meets adaptive learning: the present and future of education in nonoperating room anesthesia.
PURPOSE OF REVIEW: Nonoperating room anesthesia (NORA) and continuing education in this setting are both in a state of change. Procedures performed outside the operating room, especially those that require nuanced anesthetic care, are becoming increasingly prevalent. Numerous new innovations are transforming care across NORA settings, especially within gastroenterology, interventional and neurointerventional radiology, and electrophysiology. Concurrently, education for NORA providers is rapidly evolving, influenced by technological advancements and artificial intelligence. RECENT FINDINGS: Traditional learning, a mode of education in which each learner receives essentially the same content, is now blending with adaptive learning, a field of precision education in which dynamic educational content is curated for individual learners, timed and paced to their preferences and progress, and coupled with artificial intelligence-generated feedback. The mainstays of traditional education for NORA providers remain relevant: content created by professional societies, podcasts, textbooks, and primary literature. Yet new, artificial intelligence-driven approaches for adaptive learning have entered the scene: clinical decision-making tools, literature review and knowledge mapping assistants, online platforms for precision education, virtual reality, and synthetic patients or avatars. SUMMARY: This article explores the blending of traditional and adaptive educational modalities in NORA continuing education and discusses the implications and obstacles for the future.
23 June 2026
Read appraisal →Journal of medical Internet research
Patient Cognitive Bias in Large Language Model-Supported Health Consultations: Simulation-Based Comparative Study
BACKGROUND: Large language models (LLMs) are increasingly used by patients for health information and preliminary medical advice. In patient-facing consultations, users may present explicitly stated diagnostic preferences or symptom narratives emphasizing a preferred explanation. Such cognitively biased input constrains the diagnostic context available to the model and may systematically steer its reasoning during interactive LLM-supported health consultations. OBJECTIVE: This study aimed to quantify the impact of patient cognitive bias on LLM diagnostic performance in multiturn consultations, assess the effectiveness of prompt-based mitigation strategies and decoding temperature adjustment, and evaluate a dual-system framework for improving robustness under biased interaction. METHODS: We developed a simulated patient agent to generate both unbiased and cognitively biased consultations using 1273 medical question answering dataset United States Medical Licensing Examination cases. Six widely used LLMs of varying capacities were evaluated through 3-round, multiturn dialogues, after which each model produced a final diagnostic judgment based on the complete consultation record. Diagnostic accuracy was the primary outcome. Secondary outcomes included bias-induced accuracy decline (absolute reduction in accuracy under biased vs standard consultations) and bias-influenced error proportion (proportion of incorrect responses aligned with the patient's preferred but incorrect diagnosis). Three prompt-based mitigation strategies and 4 decoding temperature settings were tested. In addition, a dual-system framework was evaluated, in which a conversational foundation LLM conducted patient interaction and history taking (System 1), while a reasoning-oriented LLM (o1-mini) generated the final diagnostic judgment (System 2). In the foundation-only condition, the same LLM performed both interaction and diagnosis. RESULTS: Across all 6 evaluated models, cognitively biased consultations led to marked diagnostic accuracy declines of approximately 7 to 39 percentage points compared with standard multiturn consultations, whereas static single-response tests and standard consultations showed comparable accuracy. Larger deteriorations were observed in lower-capacity models, with some approaching random-guess performance under bias. Errors were frequently aligned with patient bias, with bias-influenced error proportion exceeding one-third across models, indicating systematic conformity rather than random error. Prompt-based mitigation strategies and decoding temperature reduction yielded limited and inconsistent improvements and did not reliably prevent bias-induced performance loss. By contrast, the dual-system framework substantially improved diagnostic accuracy under biased conditions, producing gains of approximately 10 to 39 percentage points across most models and recovering a large proportion of the performance lost due to bias, particularly in lower-capacity systems. CONCLUSIONS: Patient-driven cognitive bias represents an underrecognized behavioral risk in LLM-supported health consultations. Common mitigation approaches, such as prompt engineering or decoding parameter adjustment, provide limited resilience. Explicitly separating conversational interaction from deliberative diagnostic reasoning through a dual-system framework enables more robust diagnostic performance under biased input while potentially preserving patient-facing dialogue fluency by retaining the foundation LLM as the conversational component, offering a scalable design strategy for safer medical AI systems.
14 June 2026
Read appraisal →JMIR research protocols
Reducing Mis-triage in Emergency Departments (RemEDy): Protocol for Improving Triage Accuracy Through Real-time Evaluation and Artificial Intelligence
BACKGROUND: Mis-triage represents a global concern, with reported rates ranging from 15% to 33%. Understanding its causes and contributing factors is essential for ensuring patient safety. Currently, available studies have mainly focused on evaluating triage systems rather than investigating the human factors affecting triage performance. A major limitation in triage evaluation studies is the lack of standardized criteria to assess patient acuity and the absence of a clear consensus on how to measure triage accuracy. Most studies rely on retrospective data, which often fail to capture real-life clinical complexity. Therefore, the underlying causes and consequences of mis-triage remain partially understood. OBJECTIVE: This study aims to improve triage by defining the optimal triage evaluation process and identifying clinician-, patient-, and system-level factors that compromise its accuracy and safety. METHODS: Reducing Mis-Triage in Emergency Departments (RemEDy) will be a 4-phase, mixed methods project conducted across 7 Swiss emergency departments. The first phase will focus on developing a standardized triage evaluation instrument, combining evidence from a scoping review of triage evaluation processes, workshops with triage clinicians using design thinking methodology, and a modified Research and Development-University of California Delphi involving international experts and patient representatives. The second phase will prospectively implement this instrument in real time within a multicenter observational cohort study to evaluate triage performance; quantify mis-triage; and identify predictors at the patient level (eg, demographics), clinician level (eg, training), and system level (eg, crowding and length of stay). The third phase will focus on designing and validating an artificial intelligence-based decision support tool, applying multimodal models that integrate real-time triage data to enhance acuity prediction and minimize human error. The fourth phase will develop and evaluate a targeted training program, guided by the Capability, Opportunity, Motivation, and Behavior model, to strengthen triage accuracy and mitigate cognitive biases. RESULTS: The project was funded by the Swiss National Science Foundation in March 2025 (grant 10004535). At submission, the scoping review is ongoing and expected to be completed in early 2026. Development and piloting of the triage evaluation instrument will take place in 2026. A multicenter cohort study is planned between October 2026 and June 2027. The intervention study is scheduled between October 2027 and December 2028. Final results are expected in 2029. CONCLUSIONS: The RemEDy project addresses key limitations of current triage research, including the lack of standardized evaluation methods. By combining expert and clinician consensus; real-time assessment; and multilevel analysis of patient-, clinician-, and emergency department-level factors, RemEDy is expected to provide a more comprehensive understanding of mis-triage and its causes. RemEDy will establish a novel framework for real-time triage evaluation and inform the development of targeted training programs with the potential to improve triage accuracy, safety, and equity. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/92264.
28 May 2026
Read appraisal →JMIR nursing
Simulation-Based Training for Nursing Students to Improve Patient Safety: Systematic Review
BACKGROUND: Patient safety is a fundamental pillar of health care quality. Simulation-based training provides a controlled environment for nursing students to develop safety competencies and error-recognition skills before clinical practice. OBJECTIVE: This systematic review aimed to describe and characterize the simulation-based education features and modalities used to address patient safety outcomes in undergraduate nursing students, identifying the strategies that contribute to improvements in safety-related competencies. METHODS: A systematic review was conducted following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines across PubMed, Web of Science, Scopus, CINAHL, Cochrane, and Lilacs (2019-2024). Inclusion criteria focused on original studies involving undergraduate nursing students and simulation interventions measuring patient safety outcomes. Studies in languages other than English, Spanish, or Portuguese were excluded. Two reviewers independently performed study selection and data extraction. Methodological quality was assessed using Joanna Briggs Institute tools, applying a 60% quality threshold for inclusion. Results were synthesized through a narrative approach. RESULTS: A total of 20 studies from 12 countries were included. The methodological quality was high (n=14) and moderate (n=6). Findings revealed that high-fidelity simulation and virtual reality are the primary strategies used. Simulation proved effective in enhancing both technical skills (medication administration accuracy) and nontechnical skills (communication via SBAR [Situation, Background, Assessment, Recommendation] and ISBAR [Identification, Situation, Background, Assessment, Recommendation] tools, teamwork, and adverse event reporting). Key strategies contributing to safety included repetitive practice and interprofessional simulation, which significantly improved error detection and clinical judgment. CONCLUSIONS: Simulation is an essential pedagogical strategy for preparing nursing students to deliver safe care. Practical implications include the need to integrate structured simulation into nursing curricula to bridge the theory-practice gap. Future research should prioritize longitudinal designs to assess the retention of these safety skills in clinical settings and develop standardized metrics for measuring patient safety outcomes.
28 May 2026
Read appraisal →JMIR formative research
Differences in Safety Risks Across Languages in Health-Relevant Queries: Vulnerability Analysis of Large Language Model Responses
BACKGROUND: Large language models (LLMs) such as ChatGPT are increasingly used to support health-related queries and decision-making. However, these models can be "jailbroken" through adversarial prompts that bypass safety filters and elicit harmful or medically inappropriate responses. In health care contexts, such vulnerabilities pose serious risks. Understanding how jailbreak susceptibility varies across languages is essential for developing robust safeguards and promoting equitable access to safe health information. This paper may contain examples that may be deemed harmful in terms of violence, self-harm, and drug abuse. OBJECTIVE: This study aims to systematically compare and contrast the vulnerability of a health LLM for jailbreaking across 3 languages: English, Spanish, and Hindi (transliterated using the Latin alphabet), based on emoji and permutation cipher attacks. METHODS: We analyzed 1000 input prompts per language, drawn from the BeaverTails dataset, across 3 harm categories: self-harm, violence, and drug abuse. Each prompt was modified using emoji and permutation cipher techniques, resulting in 6000 input-output pairs. Model responses were evaluated by human coders to determine the success rate of jailbreak attempts across languages and cipher types. RESULTS: Hindi prompts showed the highest vulnerability, with 787 successful jailbreaks using emoji ciphers and 873 using permutation ciphers. Spanish and English followed, with lower success rates across both cipher types. Differences in jailbreak success across languages and cipher strategies were statistically significant. Additionally, attacks targeting violence-related prompts were more successful overall than those targeting drug-related or self-harm content, indicating variation in vulnerability by harm type. CONCLUSIONS: The findings of this formative study reveal that LLM safety performance varies substantially across languages and harm categories, raising concerns about equitable protection in multilingual health communication. Disparities in access to harmful content may contribute to downstream health risks. Strengthening multilingual content moderation and developing language-aware safety mechanisms are critical steps toward creating safer and more inclusive health AI systems.
28 May 2026
Read appraisal →Journal of medical Internet research
Patients' Perspectives on the Implementation of AI in Radiological Diagnostics: Focus Group Study
BACKGROUND: Rapid developments in artificial intelligence (AI) will enable its widespread use in radiological diagnostics in the near future. Patients will then be confronted with findings generated with the help of AI. Understanding patients' perspectives on the use of this technology is one of the key factors for its successful implementation. OBJECTIVE: This qualitative study aimed to gain insight into patients' reasoning about the opportunities and risks of using AI in radiological diagnosis, to identify prerequisites for its acceptance, and to identify aspects that can promote trust in AI diagnoses, especially in scenarios of high personal concern. METHODS: A total of 7 focus groups were conducted with 34 patients (n=15, 44% female participants) aged between 23 and 85 years (mean 49.06, SD 17.08 y), recruited using purposive sampling strategies. Each focus group was audiotaped, transcribed, and analyzed using the method of structured qualitative content analysis. RESULTS: Study findings show that patients are open to the use of AI in radiological diagnostics. The basic prerequisites for this are (1) scientific evidence of safe outcomes that are more accurate and faster than those without AI; (2) recognizable added value in patient care; (3) transparency in the use of AI and disclosure to the patient; (4) comprehensive, binding measures for quality assurance; and (5) the use of AI solely to support the physician. However, the results indicate that further criteria are important for patients to be willing to choose a radiologist who uses AI and to trust AI diagnoses. In situations where they are personally affected, patients fear that physicians will place too much trust in the AI result and that the physician-patient relationship will become dehumanized. Therefore, the physicians' abilities and functions that inspire trust from the patients' perspective must come into play. These include (1) an independent diagnosis by the physician that takes into account not only the clinical context but also the individuality of the patient, (2) a comprehensible explanation of the pros and cons of using AI for patients and clear communication of AI output, and (3) a humane and empathetic physician-patient relationship, which shows that the physician continues to feel responsible for the patient. CONCLUSIONS: The results of the study underscore that a high quality of the entire "sociotechnical" system is an essential prerequisite for patient acceptance of the use of AI in radiological diagnostics and for trust in AI diagnoses. The further development of AI performance must go hand in hand with the creation of framework conditions for its use that meet patients' expectations of the role of the physician and ensure a trust-building physician-patient relationship. The study provides valuable insights into how such integration of AI into radiological practice can be achieved.
26 May 2026
Read appraisal →Studies in health technology and informatics
Mapping Existing Evidence on Physicians' and Patients' Experiences with GenAI in Clinical Communication and Documentation: A Rapid Review
GenAI tools like chatbots and ambient scribes are increasingly used in clinical documentation, raising ethical and safety concerns. This rapid review maps existing evidence on experiences and attitudes of physicians and patients toward using GenAI in clinical communication and documentation. Structured searches were conducted in PubMed and Web of Science. Studies were screened using PICO criteria. Data were extracted and summarized descriptively, with high-order thematic analysis of research aims and emerging recommendations. 41 studies were included: 75.6% focused on physicians, 9.8% examined patient perspectives exclusively, and 14.6% included both. Some of the themes identified were usability, exploration of perceptions and expectations, acceptability, and ethical implications. Recommendations emphasized the need for diverse cohorts, real-world validation, and ethical considerations. Current literature was skewed toward physician perspectives. Broader, meaningful inclusion of patients is needed to support equitable and trustworthy GenAI integration in healthcare.
24 May 2026
Read appraisal →Studies in health technology and informatics
Advancing Knowledge in Evaluating the Clinical Impact of Large Language Models for Clinical Text Summarization: A Narrative Review
Large language models (LLMs) are increasingly explored for clinical text summarization from electronic health records (EHRs), where they could reduce documentation burden. While promising, errors in generated summaries may propagate misinformation, misrepresent clinical facts, and ultimately pose risks to patient safety. This review examined how the clinical impact of LLM-based summarization has been evaluated across four dimensions: utility, failure, patient safety risks, and bias. Literature was retrieved from PubMed between June 2024 and September 2025. Of the 144 retrieved studies, 32 studies were included in the analysis. Failure analysis was the most frequently reported (n=16, 50%), focusing on inaccuracies, omissions, and hallucinations, though definitions and methods varied widely. Utility analysis (n=8, 25%) examined workflow efficiency, readability, and understandability. Bias analysis (n=7, 22%) considered gender, race, and stigmatizing language. Only one study explicitly conducted patient safety risk analysis (n=1, 3%), using risk matrices. Findings indicate that evaluation frameworks are gradually moving beyond technical performance metrics; however, methodologies remain heterogeneous and lack standardization. Future research should establish consistent error taxonomies, structured risk assessment frameworks, and systematic bias detection methods to enable the safe clinical deployment of LLM-based summarization systems.
24 May 2026
Read appraisal →JMIR mental health
Automated Safety Testing and Reporting Application for Conversational Safety Monitoring of Generative AI Tools for Mental Health: Development and Validation Study
BACKGROUND: Artificial intelligence (AI)-based conversational tools are rapidly expanding within mental health care as a means of increasing access and scalability. At the same time, these systems introduce distinct safety risks arising from both user disclosures (eg, self-harm ideation) and inappropriate or inadequate AI responses. OBJECTIVE: This study aimed to develop and evaluate the Automated Safety Testing and Reporting Application (ASTRA), an external system intended to identify clinically relevant risk behaviors across entire AI-mediated mental health conversations. METHODS: ASTRA was tested on a dataset of 100 synthetic therapeutic conversations written by licensed clinicians to reflect risk behaviors and harmful responses between users and AI tools. Conversations varied in length and included both subtle and overt risk behavior examples across 8 predefined categories. Human coder consensus ratings served as the reference standard. ASTRA's classifications were evaluated across 2 prompt iterations using standard diagnostic performance metrics and agreement statistics. RESULTS: ASTRA demonstrated consistently high concordance with expert human ratings across all categories. Accuracy exceeded 0.90 for all risk behavior categories examined, with specificity uniformly high and sensitivity varying by category (range 0.55-1.00). Agreement beyond chance was substantial to almost perfect between ASTRA and human raters (κ=0.65-1.00). Detection of user self-harm indicators was particularly accurate, even in conversations where risk was expressed subtly. CONCLUSIONS: In this initial validation study, ASTRA reliably identified multiple forms of mental health-related risk behaviors at the conversation level. These findings support the feasibility of independent safety monitoring systems as a complement to AI tools used in mental health contexts and underscore the need for further evaluation using larger and real-world datasets.
21 May 2026
Read appraisal →Physiological measurement
Perioperative hypoxaemia early warning based on high-frequency waveform fusion and deep learning
Objective.The accurate prediction of impending intraoperative hypoxaemia events is paramount for patient safety. Current models relying on structural parameters exhibit limitations, including low sampling frequency, weak temporal coherence, and high data attrition rates. Consequently, long-term sequence, high-frequency waveform data are utilised in this study to construct a deep learning-based hypoxaemia prediction model, by which conventional shortcomings are addressed whilst predictive accuracy and robustness are enhanced.Approach.Respiratory, blood oxygenation, and electrocardiogram waveforms were extracted from the VitalDB database. Two processing methods were employed: extraction of waveform-derived variables to construct numerical datasets; and utilisation of the Gramian Angular Summation Field (GASF) and the four-quadrant matrix method to construct image-based datasets. Through multiple dataset construction methods, five datasets were constructed. An LSTM algorithm was employed, using a 5 min learning window and prediction windows ranging from 1 to 13 min, to conduct a comprehensive performance evaluation.Main results.Optimal performance was achieved by all models within a one-minute prediction window. Highly competitive accuracy (0.932), AUC (0.953), and stability (maximum AUC variance 0.067) were demonstrated by the 'Multimodal image model', whilst the poorest performance was exhibited by the 'structured only model'. Moreover, compared to singlemodal models, multimodal models incorporating structured parameters yield only marginal improvements. Consequently, singlemodal data derived solely from raw waveforms are suggested as sufficient for prediction requirements, by which greater application potential is held due to enhanced computational efficiency.Significance.This study systematically demonstrated the significant advantages of high-frequency waveform data. Results indicate that waveform processing methods based on the GASF and the four-quadrant matrix approach can effectively enhance model stability and performance. Furthermore, the singlemodal model relying solely on waveform data, excluding structured parameters, can meet clinical prediction requirements while reducing computational costs. This study provides novel data structures and methodologies for perioperative hypoxaemia prediction.
20 May 2026
Read appraisal →BMJ quality & safety
Patient safety measures for virtual consultations in primary care: a systematic review
OBJECTIVES: With the growing adoption of virtual consultations in primary care, the need for tailored metrics to evaluate their safety became increasingly urgent. This systematic review seeks to identify and review existing safety measures that could be used for safety evaluation of virtual consultations in primary care. METHODS: This has been conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and followed a published protocol. A systematic literature search was performed in Ovid MEDLINE/PubMed, Embase and Cochrane Library databases from 2014 to 2024. Studies comparing virtual consultations with face-to-face consultations in the primary care setting were included. An inductive thematic analysis was performed to systematically extract and group the safety measures into overarching themes, with a narrative synthesis to summarise the results. RESULTS: A total of 47 studies (31 experimental and 16 observational studies) were included (n=2 223 697 patients). All studies assessed the safety of virtual versus face-to-face consultations via one or both of the following domains: (1) factors that influence the safety of virtual consultations and (2) tangible outcomes of virtual care safety. The former were categorised into provider-related, patient-related and system-related factors. Tangible outcomes were evident through three subthemes-adverse events, health outcomes and patient perception of safety. CONCLUSIONS: This review provides a systematic synthesis of measures for the safety evaluation of virtual consultations. Further research into patient and physician perspectives is needed to identify aspects and indicators not captured in this study, followed by a consensus study to finalise safety metrics. Ultimately, having a robust methodology for safety evaluation of virtual consultations in place will enable safety monitoring, root cause analyses and safety improvement. PROSPERO REGISTRATION NUMBER: PROSPERO CRD42023464878.
20 May 2026
Read appraisal →JMIR medical informatics
Implementation and Evaluation of an Alternative Electronic Health Record Tool for Ordering Blood Products in Pediatric Oncology and Stem Cell Transplantation: Mixed Methods Analysis
BACKGROUND: Repeated blood product ordering is associated with order entry errors and potential patient harm. Traditional electronic health record order sets require repeated re-entry for recurrent transfusions, creating inefficiencies and opportunities for error, and contributing to physician burnout. Historically, we have used order sets to order blood products, which must be re-entered each time a transfusion is needed. Reusable transfusion therapy plans may address these challenges by standardizing and streamlining transfusion workflows. We conducted a pre-post study at a single pediatric academic center, evaluating the implementation of reusable transfusion therapy plans for packed red blood cells and platelets in oncology patients and those undergoing hematopoietic stem cell transplantation. OBJECTIVE: The primary outcome was to evaluate the proportion of transfusions originating from the transfusion therapy plans during the postimplementation period. Secondary outcomes included evaluating (1) the proportion of eligible patients with applied transfusion therapy plans, (2) changes in transfusion efficiency (time from laboratory result to transfusion release and administration, premedication timing, and estimated overnight pages), and (3) the impact on safety (guideline-concordant dosing, irradiated product ordering, and transfusion thresholds). We also assessed health care practitioner experience using an adaptation of the technology acceptance model survey. METHODS: The prestudy period consisted of the 1-year preimplementation, and the postperiod consisted of the 1-year post implementation. We used our institution's enterprise data warehouse (SickKids Enterprise-Wide Data in Azure Repository) to obtain demographic and transfusion details for all eligible patients. The adapted technology acceptance model survey was administered to eligible oncology clinicians. RESULTS: The preimplementation cohort had 558 unique patients who received a total of 2678 transfusions. The postimplementation cohort had 521 unique patients who received 2777 transfusions. During the postimplementation period, 59% of transfusion orders originated from a therapy plan, increasing to 71% in the final month. Compared with order sets, therapy plan-derived transfusions were released and administered significantly faster following laboratory results (P<.001). Guideline-concordant transfusion volumes increased significantly postimplementation for both packed red blood cells and platelets (P<.001), as did the ordering of irradiated blood products (P<.001). No differences were observed in pretransfusion hemoglobin or platelet thresholds between study periods. Use of therapy plans was associated with an average avoidance of 4 overnight blood product entries per night. Survey responses from nurses and providers demonstrated high perceived usefulness and ease of use, with 95% endorsing continued use. CONCLUSIONS: Reusable transfusion therapy plans improved efficiency, standardized safe ordering practices, and were highly acceptable to clinicians. This longitudinal, noninterruptive electronic health record intervention represents a scalable approach to supporting high-risk transfusion workflows in pediatric oncology.
17 May 2026
Read appraisal →JMIR medical informatics
A Multiassessment and Multiprofessional Agents Approach for Medical Chatbot Risk Estimation: Development and Evaluation Study
BACKGROUND: Assessing chatbot responses across 3 domains-medical, ethical, and legal-is essential to ensuring the safe use of artificial intelligence in health care. Although advancements in the use of large language models (LLMs) show significant improvements in evaluating question-answer datasets, such as multiple-choice medical exams, existing systems use general LLMs without incorporating specialized domain knowledge. They rely on standardized instructions without integrating real-world information, and ensemble methods such as majority voting fail to resolve disagreements among agents, resulting in misclassification and challenges in risk assessment. OBJECTIVE: This study aims to design, develop, and evaluate a synergistic approach for assessing risks associated with chatbot responses using multiassessment (MA) and multiprofessional agents (MPAs). METHODS: We designed and developed an approach consisting of MA and MPA, specifically initial assessment (MA1), which internalizes 3 roles and provides an initial risk estimation, and final assessment (MA3), which aims to reach a final consensus based on the previous assessments (MA1 and MA2), with each using 1 LLM. The verification assessment (MA2) incorporates an MPA or role-based LLM specialized agents for each risk domain (medical, ethical, and legal). We evaluated the proposed approach using the MedNLP-CHAT (Medical Natural Language Processing for AI Chat) corpus (N=226; 100 train, 126 test), covering baseline, enhanced prompt, embedding-based search, and retrieval-augmented generation (RAG). Primary metrics included macro F1-score and joint accuracy to evaluate system performance, along with CI and paired macro F1-score difference (Δ) as supporting metrics to assess the approach's effectiveness. RESULTS: The MA-MPA framework integrated with RAG achieved the highest average macro F1-score of 0.800 across risk domains and a joint accuracy of 76 (60.3%) correct predictions across all risk domains out of 126 question-answer pairs, with notable improvements over the best reported Eighteenth NII Testbeds and Community for Information Access Research Project (NTCIR-18) MedNLP-CHAT systems in the ethical (+0.252) and legal (+0.096) risk domains, while the medical domain showed a modest increase of +0.070. The MA approach contributed the largest gains, particularly from MA1 to MA2, with paired macro F1-score gains ranging from +0.176 to +0.214 across systems. The MPA approach performed better when integrated with MA and external knowledge, with paired bootstrap estimates showing a gain of +0.037 (95% CI 0.003-0.074) over baseline; however, joint accuracy gains were not evident (95% CI -2.9% to 7.7%), and gains relative to the enhanced prompt were small. Notably, MA alone achieved higher joint accuracy than RAG (62.7% vs 60.3%), indicating a metric-specific trade-off rather than consistent superiority across all metrics. CONCLUSIONS: The MA-MPA approach shows potential for improving risk estimation in chatbot responses. The results suggest that the framework is particularly useful for enhancing balanced overall performance, especially when combined with external knowledge, although the medical risk domain remains challenging. Furthermore, more specialized LLMs may further improve contextually grounded risk estimation.
17 May 2026
Read appraisal →Journal of medical Internet research
Understanding Patient-Reported Offenses in Electronic Health Records: Cross-Sectional Mixed Methods Survey
BACKGROUND: Patients' access to their electronic health record (EHR) supports their participation and satisfaction with care. Despite the benefits, some patients have been upset after reading their EHR. Additionally, health care professionals are concerned that patients, particularly those with mental health conditions, may be offended, and they have expressed a need for further guidelines on how to write EHRs. Experiences among various patient groups are essential to support the relationship between patients and professionals. However, prior studies have often focused on single patient groups or specific clinical contexts, leaving a limited understanding of differences across multiple patient groups. OBJECTIVE: This study aimed to determine whether certain patient groups are more likely to feel offended while reading their EHRs and which information is perceived as offensive and to provide a comparison across multiple patient groups using a mixed methods approach. METHODS: A cross-sectional survey was conducted via the Finnish national patient portal using a web-based patient survey, adopting a mixed methods approach. The survey included multiple-choice and open-ended questions. The total sample comprised 4681 respondents. The survey respondents were placed into 4 patient groups: those who had received care for mental health, cancer, or other conditions and those who had received no care. Associations between the type of care and patients who felt offended were estimated using multivariate binary logistic regression. Inductive content analysis (n=502) was conducted to identify information perceived as offensive in the EHR. RESULTS: The patients who had received mental health care (166/654, 25.4%) or cancer and mental health care (9/39, 23.1%) were more likely to be offended by information in their EHR compared to the other groups (cancer care: 37/375, 9.9%; other conditions care: 383/3316, 11.6%; no care: 22/206, 10.7%; other conditions care: odds ratio 0.37, 95% CI 0.29-0.46; P<.001; model A). Additionally, female patients, those with bad or very bad health conditions, and patients with bachelor's or master's degrees were significantly more likely to feel offended. Errors, the health care professionals' disrespectful language, and perceived unnecessary information were the most frequently mentioned reasons for being offended. Patients with mental health care reported more often that unnecessary information and professionals' opinions and word choices were experienced as offensive compared to other patients. CONCLUSIONS: This study contributes new knowledge by identifying differences across multiple patient groups. Although a minority of patients felt offended by their EHR, health care professionals should consider that some patients, particularly those who have received mental health care or cancer and mental health care, may be offended by specific information or word choices in their EHRs. To address this, health care professionals should receive education on how to write their notes in a neutral tone and avoid potentially offensive topics. Improving the quality of EHRs could strengthen the relationship between patients and professionals.
15 May 2026
Read appraisal →BMJ open
Implementation of safety management systems (SMS) in healthcare: a systematic review and international comparison
OBJECTIVES: Many industries where safety is a priority (eg, aviation) use safety management systems (SMSs), but evidence on their use in healthcare is needed to determine whether they could support patient safety improvement. We investigated the application of national-level SMSs to patient safety in terms of effectiveness, implementation and experience. DESIGN: Systematic review using a case study approach. DATA SOURCES: We identified patient safety organisations in each country and searched their websites. We also searched MEDLINE (in December 2023) and Embase (via Ovid), CINAHL (EBSCO) and Web of Science (in February 2024). ELIGIBILITY CRITERIA: Any evidence from five high-income countries that have publicly funded healthcare systems with universal coverage: Australia, Canada, Ireland, New Zealand and the Netherlands. We included publications on the effectiveness, implementation or experience of the patient safety approach in each country. DATA EXTRACTION AND SYNTHESIS: Included evidence was summarised and mapped onto an initial analytical framework based on analysis of SMSs in high-risk industries, enabling cross-country comparisons. Drafts were shared with experts from patient safety organisations in each country for comment. Formal quality appraisal was not possible as most evidence was non-evaluative. RESULTS: 53 publications were included, from Australia (5), Canada (7), Ireland (8), New Zealand (9) and the Netherlands (24). Only the Netherlands implemented a patient safety programme explicitly based on an SMS approach. Some aspects of patient safety in hospitals improved as a result, but there was significant variation in both the implementation of the programme and attributed outcomes.In the other four countries, the main components of an SMS were identified to some extent, along with evidence that their patient safety approaches had been influenced by concepts from high-risk industries and 'safety science' more widely. CONCLUSIONS: Emerging ideas from high-risk industries (beyond SMSs) and broader safety science have influenced all countries, reflecting increasing awareness of the need for initiatives to be context-specific to be successful. However, their implementation and impact need further evaluation. PROSPERO REGISTRATION NUMBER: CRD42023487512.
15 May 2026
Read appraisal →The Journal of clinical ethics
Ethical Responsibility in the Off-Label Use of AI in Medical Imaging
AbstractArtificial intelligence in medical imaging (AI-MI), categorized by the U.S. Food and Drug Administration (FDA) as "Software as a Medical Device," can offer significant benefit to medical care. FDA approval-represented by a "label"-indicates that the device has demonstrated safety and efficacy for a specific clinical task and population for which the device is intended. Off-label use of AI-MI outside of the task and/or population can be medically and ethically problematic, yet benefits may be reduced or missed if entirely prohibited. Ethical responsibility frameworks can assist in evaluating and establishing boundaries and opportunities in off-label use of AI-MI. In this work, we review three different models for ethical responsibility and apply them to off-label use of AI-MI. We find that a future-looking collective responsibility framework, involving stakeholders such as industry developers, healthcare organizations, clinicians, and, most importantly, patients and/or their representatives, promotes proactive and practical actions to help ensure safe and ethical use of AI-MI off-label. By adopting this approach, stakeholders can balance innovation with safeguards, advancing patient-centered care while mitigating potential harms associated with off-label AI-MI use. This framework is crucial as AI-MI becomes increasingly integral to clinical decision-making processes.
15 May 2026
Read appraisal →BMC anesthesiology
Automated end-tidal control versus manual regulation in low-flow volatile anesthesia: a systematic review and meta-analysis
BACKGROUND: In low-flow anesthesia, automated monitoring of end-tidal oxygen (etO2) and end-tidal anesthetic agents (etAA) has been favored to reduce clinicians' cognitive workload during anesthesia maintenance. This systematic review, therefore, aimed to determine the comparative efficacy and safety of automated end-tidal controlled anesthesia (etCA) versus manually controlled anesthesia (MCA). METHODS: An electronic search was conducted across four databases: PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science, and Scopus. The retrieved references were then analyzed against the studies' screening and eligibility criteria before the full articles were included in the study. The Review Manager (RevMan 5.4.1) was then used to conduct the statistical analysis of the reported outcomes. RESULTS: The electronic search retrieved 965 articles, among which 10 were included in this review. The studies included a pooled sample size of 1269 who underwent surgical procedures under low-flow anesthesia. The analyzed evidence suggests with low certainty that the type of anesthesia control may not have any significant effect on both the anesthesia duration (SMD - 0.00 min; 95% CI [-0.34, 0.34] p = 0.99) and the procedural duration MCA (SMD 0.04 min; 95% CI; 95% CI [-0.26, 0.33] p = 0.80). Moreover, we found with low certainty in the evidence that the amount of sevoflurane consumed may not be significantly different between etCA and MCA (SMD - 0.32 mLs; 95% CI [-0.94, 0.29] p = 0.31). Our narrative synthesis indicated that etCA can potentially improve anesthesia delivery by reducing the required anesthetist interventions and ensuring stable and accurately administered anesthetics. CONCLUSION: This review found very low-quality evidence suggesting that the type of anesthesia control may not significantly affect anesthesia delivery parameters, such as the amount of anesthesia consumed and procedural and anesthesia duration. The analyses, however, had significant heterogeneity, making the estimates highly unreliable. On the other hand, some studies indicate potential advantages of using etCA, such as reducing the number of interventions required during anesthesia; however, the evidence remains limited, preventing a full exploration of these advantages.
15 May 2026
Read appraisal →JMIR nursing
Exploring the Impact of Electronic Medical Record-Enabled Versus Paper-Based Systems on the Quality of Nursing Handover: Comparative Case-Study
BACKGROUND: Ineffective clinical handover has the potential to compromise patient safety and quality of care. Standardizing the handover process is a widely adopted improvement strategy intended to reduce failures of information transfer. By enabling real-time access to patient information, electronic medical records (EMRs) could address communication issues inherent to nursing handover. OBJECTIVE: This case study sought to compare the quality of nursing handover occurring at EMR-enabled sites with that occurring at paper-based sites, within a single Australian public health service. METHODS: A comparative case study design was used, using quantitative data collected from observational audits of 60 handovers and posthandover surveys conducted in EMR-enabled and paper-based ward environments. Handover quality was measured through compliance with the organization's Clinical Handover Standard and staff-reported perceptions, enabling comparison between cohorts. RESULTS: Compared with paper-based wards, EMR-enabled wards demonstrated more consistent communication of clinical alerts and risks and fewer interruptions, whereas paper-based wards showed higher rates of bedside handover and patient engagement. CONCLUSIONS: EMR implementation alone does not ensure high-quality nursing handover. EMR interface design and functionality may act as a barrier to bedside handover and patient engagement, and contribute to continued reliance on paper-based artifacts. Targeted EMR design and nursing informatics-led optimization are required to better support nursing handover as a complex and cognitively challenging communication process.
14 May 2026
Read appraisal →JMIR medical informatics
Expert Evaluation and Consensus on GPT-4o Summaries of Clinical Letters: Validation and Results of the Framework and Implementation of AI Tools Project
BACKGROUND: Large language models (LLMs) are increasingly used to summarize clinical documents; yet, automated metrics often inadequately capture clinical relevance and safety. In the initial phase of the "Framework and Implementation of AI Tools," an expert-driven, cocreated evaluation methodology was established to assess LLM-generated discharge letter summaries, combining prompt design considerations with intuitive expert appraisal. OBJECTIVE: This study aimed to quantify expert agreement and interrater reliability on LLM summaries of discharge letters, identify frequent and clinically relevant errors, and evaluate practical implications for integrating LLMs into documentation workflows. METHODS: Thirty expert-curated synthetic Dutch discharge letters were summarized. Thirty-one clinicians from Flemish care settings (1 university hospital, 2 private hospitals, and 2 general practice circles) evaluated the summaries. The evaluation framework consisted of 61 binary layout items assessing whether required sections and formatting were correctly present, 33 content items (correct or complete vs incorrect, subcategorizing missing, irrelevant, and hallucinated information), a 4-point global quality rating, and an open comment. Statistical analyses included descriptive statistics, mixed effects ordinal regression on the global score, consensus (agreement per question or letter) percentages, interrater reliability (Cohen κ, intraclass correlation coefficient [ICC], Fleiss κ, and prevalence index), and thematic synthesis of comments. RESULTS: Layout adherence was high (88%), especially in the conclusion section. The positive response rate for content was overall moderate (78%), with the best performance observed in the medical history section and the lowest performance observed in the medication section, which also showed the highest rate of hallucinations and the weakest interrater consensus. Across all sections, missing information was the most common error. Nearly 70% of global ratings were "good" or "very good." Higher positive response rates for content predicted better global scores (β=.079; P<.001), while layout and participant specialty were not relevant to global scoring. Consensus was high for the layout questions (median 96.8%, IQR 90.2%-100%) and somewhat lower for content (median 83.9%, IQR 67.7-96.8), with the lowest agreement in the medication section. Interrater agreement was moderate (median Cohen κ=0.36, IQR 0.29-0.43; range 0.07-0.56), but overall reliability was high (ICC 0.945, 95% CI 0.942-0.948), indicating strong consistency at the global level despite interrater variability. The prevalence index demonstrated that high ICC values were partly driven by the strong prevalence of affirmative responses in layout items, while content items showed more balanced distributions and lower agreement. CONCLUSIONS: Our framework offers a robust approach for evaluating LLM-generated discharge summaries, balancing usability and clinical relevance. Semantic integrity, especially regarding medication details, was identified as a key vulnerability. Perceived overall quality was driven by a positive response rate for content. High ICC values for global score, with lower item-level agreement lead toward the need for clearer, context-specific prompts and standardized evaluation criteria to reduce interrater variability. Human oversight and targeted automated checks for omissions and hallucinations are essential for safe clinical deployment.
14 May 2026
Read appraisal →Journal of medical Internet research
AI-Based Automation for Medication Reconciliation: Scoping Review
BACKGROUND: Medication reconciliation (MedRec) has the potential to improve patient safety by enhancing the continuity of medication information across settings. MedRec involves 3 core tasks: the creation of a best possible medication history, the identification of medication discrepancies among medication lists, and the resolution of medication discrepancies. While artificial intelligence (AI) has the potential to improve MedRec, existing reviews have not identified the ways in which researchers have used AI to facilitate MedRec tasks and their constituent subtasks or the level of automation achieved. OBJECTIVE: This scoping review aimed to map how previous research has applied AI to MedRec tasks and subtasks and assess the extent of automation achieved. METHODS: We searched MEDLINE, Embase, Web of Science, IEEE Xplore, and Compendex in June 2024 for studies that used AI to support a MedRec task or subtask, excluding entirely rule-based tools or studies focused on other aspects of medication management. After screening 2345 unique records, we conducted backward citation searching of studies included at the full-text stage, identifying an additional 795 unique records. We used a 4-stage model of human information processing as a structural lens to guide our considerations of automation, mapping the core tasks of MedRec onto this model. RESULTS: A total of 94 studies met the inclusion criteria. All studies addressed subtasks related to the creation of a best possible medication history. Only 2.1% (n=2) of the studies also addressed the identification of discrepancies. Thus, the highest stage of automated information processing achieved was information analysis, although most studies (92/94, 97.9%) only automated information acquisition steps. Most studies (67/94, 71.3%) used free-text clinical notes from the electronic health record, although a significant proportion (21/94, 22.3%) used images of pills or images of other medication-related items. Studies using text-based data used a variety of machine learning methods (eg, recurrent neural networks, conditional random fields, support vector machines, and transformers), whereas those that leveraged images typically used convolutional neural networks. Most studies (61/94, 64.9%) used publicly available data from benchmarking datasets (eg, n2c2 2022) and were strictly model development studies, with only 1.1% (n=1) being usability studies. CONCLUSIONS: This is the first review to consider the role of AI in the automation of MedRec tasks, offering a basis for prioritizing future development efforts. Current applications of AI to automate MedRec tasks are preliminary, with most work focusing on the extraction of medication information and limited to proof-of-concept model development. Future work should consider addressing infrastructural barriers to the AI-based automation of MedRec tasks (eg, data incompleteness in sources of medication information) and exploring approaches to automate discrepancy resolution. Beyond developing models, there is also a need to implement them in tools and evaluate them in real-world contexts.
13 May 2026
Read appraisal →Medical science monitor : international medical journal of experimental and clinical research
Capnography With Integrated Pulmonary Index for Preventing Hypoxemia During Pediatric Urologic Surgery Under Sedation: A Randomized Controlled Trial
BACKGROUND Procedural sedation and analgesia (PSA) can relieve pain and anxiety during invasive procedures. Carbon dioxide (CO₂) monitoring or integrated pulmonary index (IPI) can reduce respiratory depression in adults with PSA, but the effect of the combination of these 2 methods on intraoperative hypoxemia in pediatric urologic surgery is unclear. This study was conducted to determine whether capnography combined with IPI could lessen intraoperative hypoxemia during pediatric urinary procedures with PSA. MATERIAL AND METHODS We conducted a randomized controlled trial (ChiCTR 2300073943). Children referred for urinary procedures with PSA were randomly assigned to either a control group with standard monitoring or an intervention group with standard monitoring combined with capnography and IPI monitoring. The primary outcome was intraoperative hypoxemia. Secondary outcomes were severe intraoperative hypoxemia, airway management, and perioperative complications. RESULTS A total of 133 patients completed the analysis (67 in the intervention group and 66 in the control group). Compared with the control group, the incidence of intraoperative oxygen saturation decline was significantly lower in the intervention group (34.33% vs 56.06%, odds ratio=0.410, 95% confidence interval: 0.203-0.825, P=0.012). Simultaneously, the incidence of severe oxygen desaturation (SpO₂ <90%) was also significantly reduced in the intervention group (19.40% vs 36.40%, odds ratio=0.421, 95% confidence interval: 0.192-0.925, P=0.034). Regarding airway management interventions, the intervention group exhibited a higher frequency of jaw thrust maneuvers (1.84±1.31 vs 1.17±1.12 times, P=0.004) and a significantly reduced need for invasive positive-pressure ventilation (1.64±1.31 vs 2.39±1.48, P=0.002). No differences in other airway management techniques or perioperative complications were observed between the 2 groups. CONCLUSIONS The addition of capnography and IPI monitoring to standard monitoring reduces the risk of intraoperative hypoxemia in pediatric urologic surgery and reduces the need for aggressive ventilation through early intervention.
12 May 2026
Read appraisal →JMIR research protocols
Investigating the Implementation and Impact of AI-Assisted Fall Prevention in Hospitals: Protocol for a Multicenter, Multimethod Observational Study in Sweden (SAFE)
BACKGROUND: Artificial intelligence (AI) has the potential to enhance patient safety, particularly in the prevention of in-hospital falls. Recent advances in sensor-based AI systems allow for the analysis of complex, multimodal data to generate real-time alerts, enabling health care professionals to intervene before a fall occurs. By shifting from reactive responses to proactive risk management, these technologies may enable reductions in fall incidence and improvements in care outcomes. As a result, hospitals across Europe are increasingly adopting such systems. Nevertheless, empirical evidence concerning their routine implementation remains limited, particularly concerning their impact on patient safety, clinical workflows, and the usage of health care resources. Addressing these gaps is essential for effective and sustainable integration into hospital care. OBJECTIVE: This paper outlines the protocol for the multicenter, multimethod project SAFE (Safe AI-Assisted Fall Prevention Through Evidence), which investigates the implementation and impact of AI-assisted fall prevention in Swedish hospitals. METHODS: The research project is a collaboration between Halmstad University and hospitals in the Västra Götaland Region (VGR) and will, during 2026-2028, investigate an ongoing large-scale AI system implementation in VGR hospitals, covering up to 2400 patient beds. Using surveys, interviews, observations, and a retrospective study, it will track the implementation and impact over time. Two learning laboratories involving patients, their relatives, and health care professionals will be conducted to codevelop strategies for the implementation of AI-assisted fall prevention. RESULTS: The project will provide evidence-based insights and practical guidance on AI-assisted fall prevention. The findings will be relevant not only to patients, health care professionals, and hospital organizations, but also to policymakers and stakeholders involved in the digital transformation of health care. CONCLUSIONS: Although VGR serves as the primary research setting, the project's results will inform future similar initiatives in Sweden and offer transferable lessons for other health care systems internationally. This study will contribute to the evidence base on AI-assisted fall prevention in health care, supporting the responsible and scalable integration of such systems across diverse health care environments.
11 May 2026
Read appraisal →BMJ open
Potential harms of social prescribing: a global umbrella review and dark logic model
OBJECTIVES: Social prescribing has provided a lifeline to people, capturing the importance of quality support offered via community connections. It has not been exempt from difficulties, resulting in low-quality evidence on effectiveness in part due to high drop-out rates and lack of controls. While these have been the focus of recent research, less attention has been paid to the safety implications of social prescribing. This review aims to understand safety implications of social prescribing interventions and to build a dark logic model of how harms are produced. DESIGN: Using review of review (umbrella) methodology, we searched nine databases to June 2024. DATA SOURCES: Medline/Ovid, Embase, PsycINFO, Cochrane Library/Wiley, Web of Science and Scopus) were searched from 1 January 2010 to 3 June 2024. ELIGIBILITY CRITERIA: Included reviews were systematic/scoping/narrative reviews or meta-analyses, which synthesised primary data collected from any person in receipt, involved in delivery or the commissioning of social prescribing interventions. The context of the review was social prescribing interventions/services based in primary healthcare (statutory) or third sector (non-statutory) in any country. Only reviews that were published in English and in peer-reviewed journals were included. DATA EXTRACTION AND SYNTHESIS: Two independent reviewers extracted data from included reviews using the Typology of Harms Framework (which includes five categories of harms associated with interventions: physical, psychological, group/social, equity and opportunity harms) and to build a dark logic model to understand what contributes to these harms. RESULTS: Sixteen reviews were included, reporting on social prescribing research including a link worker. Of the identified harms, we found that opportunity harms (harm related to the cost, inappropriate or ineffective interventions) were most reported. There was also evidence extracted to suggest plausible psychological, equity (impact caused by inequity in provision, delivery or access) and group/social harms (impact that overly or inadvertently excludes a person). No physical harms were identified. CONCLUSIONS: Social prescribing, as with any delivery of care, has the potential to cause harm. We identified a range of potential harms (psychological, group/social, equity or opportunity harm) from social prescribing; however, it is unlikely to be an exhaustive list. We provide two clear outcomes: (1) the need for robust design of social prescribing research, including collection of data on the incidence and prevalence of harms, (2) recognition of the potential for harm from social prescribing and to address these where practicable.
11 May 2026
Read appraisal →Journal of wound care
Incidence and risk factors of medical adhesive-related skin injury among ICU patients: a systematic review and meta-analysis
OBJECTIVE: To summarise the evidence regarding the incidence and risk factors of medical adhesive-related skin injury (MARSI) among patients in the intensive care unit (ICU). METHOD: A systematic review and meta-analysis was undertaken using articles available in nine databases: PubMed; the Cochrane Library; Web of Science; Embase; CINAHL; Chinese Biomedical Literature Database; China National Knowledge Infrastructure Database; Wanfang Database; and Weipu Database, from inception until 30 June 2024. Additional studies were identified by examining the references of relevant articles. The Stata 16.0 statistical platform (StataCorp LLC, US) was used for analysis. Study quality was assessed using the Newcastle-Ottawa Scale and the Agency for Healthcare Research and Quality standards. RESULTS: Of 14,743 titles screened, 19 studies involving 5218 patients were included in the systematic review. The pooled incidence of MARSI was 21% (95% confidence interval (CI): 14%, 28%), and the incidence of MARSI among neonatal ICU, paediatric ICU and adult ICU was 15% (95% CI: 6%, 26%), 48% (95% CI: 35%, 60%) and 16% (95% CI: 11%, 22%), respectively. Significant risk factors in adult ICUs included oedema (odds ratio (OR): 8.144; 95%CI: 3.267, 20.304), Braden scale score (OR: 0.733; 95%CI: 0.641, 0.837), length of ICU stay (OR: 1.072; 95%CI: 1.026, 1.120), mechanical ventilation (OR: 3.698; 95%CI: 1.900, 7.199), sedative use (OR: 6.022; 95%CI: 2.859, 12.687), Nutritional Risk Screening 2002 score (OR: 4.241; 95%CI: 1.984, 9.065) and hypoalbuminaemia (OR: 4.321; 95%CI: 1.967, 9.488). CONCLUSION: This study found that MARSI is prevalent among ICU patients and is associated with multiple risk factors. Improved strategies for identifying and clinically managing MARSI in ICU patients are needed. Nevertheless, large-sample and multicentre studies are still warranted to validate the present study's findings.
10 May 2026
Read appraisal →Artificial intelligence in medicine
Size doesn't matter: Assessing the trustworthiness of large language models in medical contexts: A focus on epidural information retrieval.
BACKGROUND: Since the release of ChatGPT, numerous LLMs have emerged, providing easy access to information without the need for technical expertise. However, relying on these systems can influence important life decisions, such as the choice to use epidural analgesia during childbirth. Epidural analgesia is widely regarded as the "gold standard" for pain relief during childbirth. However, limited access to anaesthesiologists and gaps in knowledge may prompt individuals to seek information from unverified sources, including AI systems. Misinformation in this area can discourage the use of effective analgesia, highlighting the need to assess the accuracy of LLM-generated content. OBJECTIVE: To evaluate the reliability of LLM-generated information regarding epidural analgesia. METHODS: We posed 10 standardized questions about epidural analgesia to 12 LLMs, each question reformulated 10 times in both Spanish and English, resulting in 2400 responses. Two anaesthesiologists were involved in the assessment process. One expert performed the initial ratings, while the second independently verified the evaluations assessing the outputs using an extended SERVQUAL framework. RESULTS: ChatGPT performed best, followed by Gemini 2. Medium-sized models, such as Phi-3 and OpenChat, outperformed several larger models like Llama-2 or Llama-3, challenging the notion that "bigger is better" and offering potential advantages in low-resource settings (e.g., Phi-3 outperformed Llama-2 with an average increase of 81% across all metrics). Specialized models did not show superior performance. Except for ChatGPT, English responses were generally more reliable than Spanish, with some Spanish outputs incoherent. ChatGPT also exhibited the least variability between responses. CONCLUSIONS: Despite promising performance, LLMs display limitations in medical contexts. Collaboration between national and international medical societies is crucial to develop evidence-based resources to guide LLM training and improve information trustworthiness.
10 May 2026
Read appraisal →Healthcare management forum
Navigating Privacy in Health Data Sharing: A Patient-Centric Approach to Health Information Exchange
Personal Health Information (PHI) sharing through Health Information Exchange (HIE) enhances patient safety in Canada, yet not all provinces and territories voluntarily disclose PHI on safety incidents to federal and pan-Canadian surveillance systems. A frequently cited barrier by healthcare organizations for HIE between different interoperable health databases is patients' and families' concerns for their privacy. This explorative qualitative study examined patients' and families' attitudes toward PHI sharing, including its secondary use in patient safety events. Rather than expressing reluctance, participants described support for HIE when privacy safeguards, such as defined sharing purposes and anonymous formats, were in place. These findings present a significant opportunity for health leaders and data custodians to use the research findings to create a patient-centric framework for the HIE of PHI.
6 May 2026
Read appraisal →European journal of pediatrics
Evaluating the impact of simulation on patient outcomes in pediatric and neonatal critical care: a systematic review and critical appraisal of the evidence
UNLABELLED: While the literature widely supports a positive relationship between simulation-based education (SBE) and learning outcomes, its impact on patient outcomes remains less clearly understood. Our aim was to comprehensively survey the landscape of simulation-based research (SBR) that evaluates patient outcomes (Kirkpatrick Level 4) in the context of pediatric and neonatal critical care. A systematic search of MEDLINE, Cochrane Library, CINAHL, and ERIC was conducted through December 2024. We included RCTs, non-RCTs, and before-after studies focusing on Kirkpatrick Level 4 outcomes. Quality was appraised using the RoB 2 and ROBINS-I tools. A total of 2481 articles were screened. Sixteen studies met the inclusion criteria. Most were prospective (63%) and single-center (69%), with only three RCTs (19%). All included studies (100%) reported Level 4 outcomes; however, a critical distinction emerged between system-level and patient-level results. While 75% (12/16) of studies reported positive impacts on Level 4a (system processes), such as protocol adherence or response times, only 50% (8/16) demonstrated significant improvements in Level 4b (direct patient outcomes), such as mortality or complication rates. In addition, all included studies were judged to have a high or critical risk of bias. CONCLUSIONS: Although SBE effectively optimizes clinical processes (Level 4a), a significant evidence gap remains regarding its direct impact on patient health (Level 4b). Current SBR often lacks the methodological rigor-specifically the use of randomized designs-needed to isolate SBE as the primary driver of clinical improvement. Future research must prioritize high-quality, multicenter trials targeting patient-oriented metrics to bridge this translational gap. WHAT IS KNOWN: • SBE has been extensively developed in pediatric and neonatal critical care settings; however, its impact on healthcare professional training and on patient outcomes remains less well understood. WHAT IS NEW: • There is a scarcity of rigorous Level 4 evidence linking SBE to patient outcomes in pediatric and neonatal critical care. • We provide a methodological roadmap advocating for mixed-methods and multicenter designs to bridge this clinical evidence gap.
1 May 2026
Read appraisal →Scientific reports
Intelligent virtual agents in psychotherapy: a safety evaluation across high-risk mental health scenarios
The growing burden of mental illness and limited access to evidence-based psychotherapy have increased interest in artificial intelligence (AI)-driven conversational agents as potential supports for mental health care. In this exploratory pilot study, we examined the safety and feasibility of an intelligent virtual agent (IVA) designed to simulate psychotherapeutic interactions, with a focus on high-risk situations involving suicidality and substance use. Two licensed psychotherapists engaged in scripted interactions with the IVA across 12 predefined scenarios addressing suicidality and substance abuse. The IVA was powered by GPT-4omni and embedded in a Unity-based avatar. After each interaction, testers evaluated acceptance, usability, and human-robot interaction. Two independent psychotherapists rated the IVA's responses using a structured scale assessing guideline adherence, risk recognition, help provision, de-escalation, and empathy. No real patients were involved; all interactions were simulated for safety testing purposes. The IVA showed preliminary indications of good usability and generally empathic responses. However, problematic responses occurred in 29% of conversations, with 12.5% rated as highly critical. Responses rated as "critical" or "highly critical" referred to outputs that failed to provide adequate support, showed insufficient risk recognition, or included ethically problematic suggestions. Key concerns included inadequate recognition of risk, normalization of substance use, and insufficient referral to crisis resources, particularly in scenarios involving underage alcohol access and suicide-related inquiries. In this small, expert-based pilot safety evaluation, the findings suggest that although AI-based agents may improve access to mental health support, rigorous safety evaluation, clinical oversight, and robust safeguards are essential prior to clinical deployment. No clinical conclusions can be drawn from this simulated study.
1 May 2026
Read appraisal →BMJ health & care informatics
Omission and hallucination prevalence of clinical guidelines in diagnostic large language model outputs
OBJECTIVE: Meaningful assessments of how large language models (LLMs) incorporate clinical guidelines require large-scale testing over many queries. Here, we evaluate the prevalence of clinical guideline omissions and hallucinations in a large sample of diagnostic LLM outputs. METHODS: We used simulated case vignettes and zero-shot prompting to generate diagnostic outputs and rationales from GPT-4.1 and DeepSeek-V3. English case vignettes were created for hypercholesterolaemia and type-2 diabetes mellitus. Each vignette contained identical medical information, while sociodemographic characteristics varied in terms of sex, ethnicity and location. We calculated the prevalence of existing and hallucinated clinical guidelines in LLM outputs across disease, LLM and sociodemographic characteristics. RESULTS: We analysed a total of 12 197 LLM outputs, which quantifies three hazard areas: omissions (up to 97% for DeepSeek-V3 and 46% for GPT-4.1), hallucinations (up to 9%) and inconsistencies (guideline citation rate ranging from 0% to 78.39% across sociodemographic vignettes). Omission and hallucination rates were generally similar across vignettes with different sex or ethnicity data, yet were particularly sensitive to patient location. DISCUSSION: This study highlights significant variability in clinical guideline prediction across two different diseases, three different sociodemographic variables and two LLMs, even when the LLMs were instructed by identical prompts, establishing clinical guideline prediction in LLM outputs as a stochastic event. CONCLUSION: The stochastic nature of LLMs creates a unique challenge for evidence generation and clinical deployment. Being able to measure and capture this stochasticity within high-quality research designs will be a prerequisite to advancing the responsible deployment of LLMs in healthcare.
27 Apr 2026
Read appraisal →BMJ health & care informatics
Does the accuracy of medication administration documentation improve with electronic medication systems? A stepped-wedge cluster randomised trial.
OBJECTIVES: Accurate documentation of medication administration is crucial for patient safety. This study compares the accuracy of documentation between paper-based and electronic medication management (eMM) systems. METHODS: A secondary analysis of data from a stepped-wedge cluster randomised trial of an eMM implementation was conducted. The eMM module allowed electronic prescribing, recording of drug dispensing, drug administration and medication reconciliation and monitoring. Medication administration processes were observed in a tertiary paediatric hospital in Sydney, Australia. Trained nurse observers recorded details of medication administrations which were then compared with the information documented by nurses about the administrations. RESULTS: 5095 administrations were analysed. With eMM compared with paper, significantly fewer administrations had no time documented (adjusted OR (aOR) 0.15; 95% CI 0.07 to 0.34; p<0.001), but documented details were more often inaccurate (aOR 4.6; 95% CI 3.3 to 6.6). Fewer doses were administered from incorrect prescriptions with eMM (aOR 0.25; 95% CI 0.18 to 0.36). CONCLUSION: eMM improves certain aspects of documentation but introduces new challenges, particularly in accurately recording administration details. Further investigation is needed to optimise these systems. TRIAL REGISTRATION NUMBER: Australian New Zealand Clinical Trials Registry 370325.
27 Apr 2026
Read appraisal →Journal of medical Internet research
The Real-Time Support Role of Augmented Reality Technology in Shared Decision-Making in Neurosurgery Under the SEGUE Framework: Randomized Controlled Trial
BACKGROUND: Preoperative risk communication is essential for shared decision-making (SDM) in neurosurgery; however, conveying complex neuroanatomy and surgical risks using traditional verbal explanations can limit understanding and contribute to dissatisfaction and medicolegal disputes. Augmented reality (AR) may provide patient-specific, interactive 3D visualization to support these conversations. OBJECTIVE: This study evaluates whether AR-assisted preoperative risk communication improves objective understanding and other SDM-related outcomes, compared with communication supported by a conventional physical anatomical model within a standardized SEGUE-informed protocol. METHODS: A prospective, single-center, randomized controlled trial was conducted with 62 neurosurgery communication recipients (patients when capable; otherwise, a legally authorized representative [LAR]). Patients were stratified by planned surgical approach (frontal, parietal, and occipital) and, within each stratum, were randomized into an experimental AR group and a control physical-model group. The primary outcome was postsession objective understanding, assessed by a multiple-choice knowledge questionnaire. Secondary outcomes were subjective understanding, communication satisfaction, pre-to-post anxiety changes, communication duration, and neurosurgeons' communication skills from video recordings. RESULTS: Of the 67 individuals screened, 62 communication recipients were enrolled and completed all assessments (patients, n=30; LARs, n=32; AR, n=32; and control, n=30). Objective understanding was higher with AR than with the physical model in the prespecified pooled comparison (P=.01). Communication satisfaction was also higher with AR (P<.001). There were no clear between-group differences in subjective understanding (P=.41), anxiety changes (ΔState-Trait Anxiety Inventory [ΔSTAI] Y-1, P=.37; ΔSTAI Y-2, P=.84), in-session face-to-face communication duration (excluding any presession AR technical preparation time; P=.73), or SEGUE scores (P=.60). Exploratory stratified analyses suggested larger comprehension gains with AR in the parietal and occipital approach strata. CONCLUSIONS: In a standardized preoperative SDM conversation, an integrated visualization-support package combining AR, patient-specific modeling, and interactivity improved neurosurgical decision makers' objective understanding and satisfaction without prolonging in-session, face-to-face communication duration. Larger multicenter trials with longer-term outcomes are warranted to confirm effectiveness and to evaluate implementation and cost considerations. TRIAL REGISTRATION: ISRCTN Registry ISRCTN11483487; http://www.isrctn.com/ISRCTN11483487.
20 Apr 2026
Read appraisal →BMJ open
Report cards and outcome measurements to improve the safety of surgical care (North America): an updated systematic review from Making Healthcare Safer IV
OBJECTIVES: In the USA, an estimated 40-50 million operations are performed annually, with high rates of adverse events. Since the 1980s, report cards have been used for outcome measures and to improve safety of surgical care. As part of Making Healthcare Safer IV-an initiative aimed at publishing evidence-based reviews as they are completed to help healthcare leaders, researchers and policymakers act more quickly on evidence-supported practices-we performed an updated review on the certainty of evidence on patient safety practices related to the use of surgical report cards and outcome measurements. DESIGN: Systematic review using the Grade of Recommendations Assessment, Development and Evaluation (GRADE) approach. DATA SOURCES: PubMed, Web of Science, Scopus and the Cochrane Library were searched from November 2011 to May 2023. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included primary research studies (randomised control trials or observational studies with a comparison group, including pre-post studies) or observational studies that investigated a surgical report card in adult or paediatric surgical patients at the hospital or surgeon level in inpatient or outpatient settings. Excluded studies included: narrative reviews, scoping reviews, editorials, commentaries, abstracts, studies that measured only patient knowledge or levels of engagement or studies using local surgical dashboard data. DATA EXTRACTION AND SYNTHESIS: Screening and eligibility were done in duplicate, while data extraction was done by one reviewer and checked by a second reviewer. Specific items in the Risk Of Bias In Non-randomised Studies - of Interventions tool and a modification of the National Institutes of Health Tool were used to assess for bias in studies. Two reviewers assessed each study for risk of bias. A modified version of the GRADE framework was used to assess the certainty of evidence. RESULTS: We identified 19 studies that met the inclusion criteria: 13 primary research studies and 6 descriptive studies of surgical collaboratives. Of the primary studies, nine used a pre-post or longitudinal study design and four used a regression discontinuity or concurrent controlled design. Seven of the studies were about the American College of Surgeons National Surgical Quality Improvement Project. Five studies were from single institutions and the remainder included nine to greater than 700 hospitals. Pre-post studies of report cards that prompted quality improvement (QI) programmes all reported improvements in outcomes, longitudinal studies reported benefits in some but not all outcomes and one in four controlled before-and-after studies reported a statistically significant mortality benefit. All studies, except for one, were at moderate or high risk of bias. Six collaboratives were identified with preliminary data. CONCLUSIONS: Based on the above evidence, the theoretical rationale and parallel evidence in other settings, we judged that it was moderate certainty that report cards and outcomes measurements can improve surgical outcomes. However, given the evidence from studies where report cards were actively linked to institutional QI initiatives, we recommend that outcome data must be paired with actionable QI efforts to meaningfully improve patient outcomes.
19 Apr 2026
Read appraisal →JMIR medical informatics
Quality of Clinical Notes Created by Ambient Listening Generative AI: Pragmatic Prospective Pilot Study
BACKGROUND: Physicians routinely document specifics of patient encounters in clinic visit notes, a critical but potentially time-consuming task. Ambient listening artificial intelligence (AI) technology is being integrated into clinical workflows to reduce documentation burden by creating draft visit notes. While this technology is promising, it is not perfect, and the potential for patient harm needs to be understood and mitigated. We developed and piloted an efficient, standardized approach to evaluating AI-generated notes for safety concerns in ambulatory care visits. OBJECTIVE: The objective of this quality improvement project was to develop and pilot an efficient, standardized, and scalable approach to evaluating AI-generated notes for safety concerns in ambulatory care visits. METHODS: During a 2-month pilot (July to August 2024), 31 physicians across multiple specialties used an ambient listening AI scribe to assist with the creation of 7545 clinic notes. A novel survey instrument was developed to assess note quality, focusing on 4 error types: accidental inclusions, accidental omissions, hallucinations, and bias. Physicians evaluated 356 (4.7%) AI-generated notes. Where an error was present, physicians rated its severity based on its potential to cause patient harm if it was not corrected, on a 0 to 5 scale. Additionally, a vendor-reported metric on the percentage of note content edited by physicians was analyzed. RESULTS: Of the 356 evaluated notes, accidental omissions were the most frequent error (n=64, 18%), followed by hallucinations (n=41, 11.5%), and accidental inclusions (n=33, 9.3%). Bias was rare (n=4, 1.1%). Most (119/142, 83.8%) errors were rated as mild to moderate (severity 1-3), with only 19 (5.3%) notes containing errors rated as posing serious or imminent risk (severity 4-5). Editing metrics across all AI-created notes showed a median of 9.0% (IQR 2.5%-21.9%) of AI-generated words were changed, with 14.9% (143/960) of notes left entirely unedited. Physician editing practices varied widely, with average percentages of AI-generated words changed ranging from 1.9% to 69.3% (median 9.0%, IQR 2.5%-21.9%). CONCLUSIONS: AI-generated clinical notes were generally of high quality, with 94.7% (337/356) free from significant errors. However, because a small number contained errors that carried the risk of serious harm if not corrected, careful clinician review of notes remains imperative. Prior to deploying an AI scribe, organizations should pilot the technology and include an efficient review process to understand the nature and type of errors common at their organization. This pilot provides a scalable model for other health systems seeking to implement AI scribe technology responsibly.
18 Apr 2026
Read appraisal →JMIR formative research
Fact-Checking Large Language Model Responses to a Health Care Prompt: Comparative Study
BACKGROUND: Large language models use machine learning to produce natural language. These models have a range of potential applications in health care, such as patient education and diagnosis. However, evaluations of large language models in health care are still scarce. OBJECTIVE: This study aimed to (1) evaluate the accuracy and efficiency of automated fact-checking by 2 large language models and (2) illustrate a process through which a large language model might support a patient in redrafting a prompt to include key information needed for patient safety. METHODS: A parallel comparison of 2 large language models and 3 human experts was conducted. A clinical scenario was devised in which a woman aged 23 years questions the safety of retinoid treatment for acne by sending prompts to 2 large language models (GPT-4o and OpenBioLLM-70B). GPT-4o and OpenBioLLM-70B were asked to suggest improvements to the patient's initial prompt to elicit key information for clinical decision-making. After the patient sent the revised prompt to the large language models, the models were then asked to fact-check the final response. To test the generalizability of automated fact-checking, a set of 20 clinical statements on disparate topics, mostly related to drug indications, contraindications, and side effects, was developed. The large language models also fact-checked these 20 medical statements. The results were compared against the evaluations of 3 clinical experts. The outcome measures were as follows: (1) percentage of accuracy of automated fact-checking, (2) time to complete fact-checking, and (3) a binary outcome for prompt redrafting (advising the patient to revise her prompt by naming her acne medication to address safety concerns). RESULTS: For the scenario of a patient with acne, GPT-4o and OpenBioLLM-70B both had 86% agreement with the clinical experts' fact-checking. The large language models did not consistently convey the urgency of discontinuing isotretinoin treatment when pregnancy is suspected. In addition, the models did not adequately convey the importance of folic acid supplementation during pregnancy. For the set of 20 medical claims, GPT-4o fact-checking had 100% agreement with that of human experts, whereas OpenBioLLM-70B had 95% agreement. OpenBioLLM-70B diverged from human experts and GPT-4o on 1 question related to pediatric use of antihistamines. The expert fact-checks took a mean time of 18 (SD 3.74) minutes, GPT-4o took 42 seconds, and OpenBioLLM-70B took 33 minutes. The GPT-4o responses for the acne scenario had some inconsistencies but zero fabrication and no obvious omissions. In contrast, OpenBioLLM-70B omitted 1 key information item needed for patient safety. CONCLUSIONS: GPT-4o can interact with patients to improve the quality and comprehensiveness of the information contained in health-related prompts. GPT-4o and OpenBioLLM-70B can conduct efficient fact-checking that is close to the level of accuracy of human experts. Human experts need to perform additional checks for accuracy and safety.
17 Apr 2026
Read appraisal →BMJ open
Challenging history taking encounters: a systematic review, meta-analysis and phenomenological framework
OBJECTIVES: All physicians will experience challenging history taking encounters, where communication is impaired and negatively impacts the diagnostic process. The aims of this systematic review were to (1) undertake a meta-analysis of the frequency of challenging encounters; (2) collate adverse outcomes of challenging encounters; (3) identify underlying causes of challenging encounters; (4) identify strategies to deal with different challenges; and (5) align these strategies with our published phenomenological framework of history taking challenges. DESIGN: This was a systematic review and meta-analysis of prevalence data adhering to the Preferred Reporting Items for Systematic reviews and Meta-Analyses and the Meta-analyses of Observational Studies in Epidemiology guidelines. DATA SOURCES: A literature search in MEDLINE, Embase and Cochrane databases was performed on 12 July 2020, and updated on 4 August 2025, focusing on challenging history taking encounters in any clinical setting. ELIGIBILITY CRITERIA: Articles reporting on the frequency, adverse outcomes, causative factors or strategies used to address challenges in the history taking process in any clinical area of medicine. DATA EXTRACTION AND SYNTHESIS: Factors associated with challenging history encounters (causative or consequential) were categorised using inductive coding and referenced to a phenomenological framework. Meta-analysis was used to estimate the prevalence of history taking encounters using a restricted maximum likelihood model with τ 2 and I 2 as tests for heterogeneity and funnel plot with Egger's test for publication bias. RESULTS: 73 articles were included in the analysis. The overall prevalence of challenging history taking encounters was 19.5% (95% CI 14.2% to 24.7%). Adverse outcomes of patient dissatisfaction (level 1 evidence) and diagnostic uncertainty (level 3 evidence) were identified. Factors associated with (n=22) and strategies to mitigate challenging encounters (n=13) were categorised. Correlation of factors and strategies with a phenomenological approach created a framework to assist novice history takers in approaching such circumstances. CONCLUSIONS: Challenging history taking encounters are common. Little is known of the relative importance of factors associated with challenging history taking encounters or the impact of suggested strategies. Many of the suggested strategies to facilitate meaningful communication in these situations involve a departure from standard history taking. More research is required to better define the nature of challenges encountered in history taking with a view to develop better educational models for trainee physicians.
15 Apr 2026
Read appraisal →PDA journal of pharmaceutical science and technology
The European Qualified Person: Legal Accountability, Quality Leadership, and AI Integration in Modern Pharmaceutical Manufacturing: Navigating Complex Therapies and AI-Supported Oversight to Safeguard Patient Safety
The European Qualified Person (QP) holds unique legal responsibility for certifying batch release, ensuring patient safety, and maintaining regulatory compliance. Modern pharmaceutical manufacturing-especially in biologics, advanced therapy medicinal products (ATMPs), and personalized medicines-generates complex, multi-site, data-rich environments that challenge traditional oversight. Artificial Intelligence (AI) offers predictive analytics, anomaly detection, and trend recognition to support decision-making but cannot currently replace the QP's scientific judgment or legal accountability. The forthcoming Annex 22 introduces Human-in-the-Loop (HITL) frameworks that embed human oversight within AI-supported processes, aligning technological advancement with regulatory control. This review explores evolving QP responsibilities, core and emerging competencies, AI integration within Quality Risk Management (QRM) and Good Manufacturing Practice (GMP) frameworks, and ethical considerations, illustrated with practical case studies. By developing AI literacy and applying HITL oversight, QPs can more effectively translate technological potential into transparent, science-based, and patient-centered batch-release decisions.
12 Apr 2026
Read appraisal →PloS one
Applying machine learning and natural language processing to patient safety event reports: Identifying patterns of cardiovascular diagnostic errors
OBJECTIVE: To explore the utility of natural language processing (NLP) and machine learning (ML) techniques to identify unsafe conditions leading to cardiovascular diagnostic errors using patient safety event (PSE) reports data. METHODS: PSE reports from January 2016 to August 2021 from a multi-hospital healthcare system in the mid-Atlantic region of the United States were included in this study. To have the true cardiovascular diagnostic errors labels for PSE reports, each individual PSE report was manually reviewed to find clinician-reported narratives describing current definitions of cardiovascular diagnostic errors. The PSE reports which contained cardiovascular diagnostic errors-related narratives were labeled as one and zero otherwise. Four binary ML models were employed to identify cardiovascular diagnostic errors narratives and common features from annotated PSE reports data: (1) simple logistic regression, (2) elastic net, (3) XGBoost, and (4) deep neural networks. RESULTS: XGBoost outperformed the rest of the models in identifying cardiovascular diagnostic errors -related reports and achieved high performance metrics on the testing data (AUROC = 0.914, specificity = 0.982, PPV = 0.866, accuracy = 0.929, F-1 score = 0.738, and AUPRC = 0.783). Pacemaker emerged as a significant signal for cardiovascular diagnostic errors in our PSE reports. Our analysis demonstrated that ordering MRI for patients with pacemaker was a frequent theme among cardiovascular diagnostic errors-related PSE reports containing the word pacemaker. Order, EKG, cardiac, and chest were the five most important features in identifying cardiovascular diagnostic errors events. These words were utilized in explaining heart conditions, associated care process, and related safety incidents. CONCLUSIONS: Findings from our study demonstrates the feasibility of ML and NLP techniques in identifying cardiovascular diagnostic errors-related reports among the existing PSE data sources. However, validation in external healthcare systems is needed before broader application.
11 Apr 2026
Read appraisal →JMIR medical informatics
Code-Based Versus AutoML Methods for Pill Recognition in Clinical Settings: Comparative Performance Study
BACKGROUND: Visual identification and verification of medications during dispensing and administration are prone to human error, particularly in high-pressure and high-volume clinical settings. Misidentification can lead to medication errors, posing risks to patient safety and placing a burden on health care systems. Recent advances in computer vision and object detection offer promising solutions for automated solid oral dosage form (pill) recognition. However, comprehensive studies comparing code-based and no-code (automated machine learning [AutoML]) approaches for pill recognition are lacking. OBJECTIVE: This study aimed to evaluate and compare the performance, cost, usability, and deployment feasibility of pill recognition models developed with Ultralytics YOLO11 and 3 cloud-based AutoML platforms (Amazon Rekognition Custom Labels, Google Vertex artificial intelligence [AI] AutoML Vision, and Microsoft Azure Custom Vision) using multiple datasets, including real-world clinical images. METHODS: Five training subsets of increasing size (1230, 3450, 7380, 14,400, and 26,880 images) from 30 commonly dispensed medications were used to train models on YOLO11 and 3 AutoML platforms. Models were evaluated on 6 datasets from different environments: clinical images from 3 hospitals, a verification dataset, a laboratory dataset, and an exhaustive testing set. Performance metrics, including accuracy, precision, recall, and mean average precision, were calculated. We evaluated the impact of training data size on performance and benchmarked training time, platform costs, and limitations. RESULTS: No single platform dominated across all test environments. On the verification dataset (optimal conditions), accuracy ranged from 80.83% (YOLO11) to 91.60% (Google Vertex AI) when trained with the full training dataset. YOLO11 showed consistent performance improvement with increasing training data (accuracy: 63.06%-80.83%) and achieved near-perfect precision and mean average precision scores (0.95-1.00). Google Vertex AI reached above 90% accuracy on 3 training subsets but showed unpredictable declines. Amazon Rekognition maintained near-perfect precision (0.92-1.00) but had the highest false negative rates (up to 0.74), missing many pills. Custom Vision demonstrated steady performance improvements (77.08%-85.62% accuracy) but lagged behind other AutoML platforms, probably due to its older YOLOv2-based architecture. On clinical datasets, accuracy fluctuated (20.62%-90%) depending on the dataset and platform. Training costs and time varied: YOLO11 (open-source), Microsoft Azure (US $9.50-US $28.60, allowed user-predefined training duration), Google Vertex AI (US $69.30 with consistent 2.5-3-hour training times), and Amazon Rekognition (US $5.43-US $43.89 with size-dependent training time scaling, reaching nearly 40 hours on the full 26,880-image dataset). CONCLUSIONS: Each platform offers distinct advantages and trade-offs: YOLO11 provides the highest flexibility and lowest platform costs but requires technical expertise, while AutoML platforms can offer high performance at a higher cost but with limited user control, introducing unpredictability. The performance variations demonstrate that successful clinical deployment requires careful platform selection based on specific performance requirements, budget constraints, and available technical resources, followed by rigorous validation using real-world, representative data to ensure patient safety in clinical workflows.
11 Apr 2026
Read appraisal →Scientific reports
Effects of different inflatable heater temperature settings on early postoperative recovery after hysteroscopic surgery: a randomized clinical trial
To investigate the effect of perioperative temperature settings of an inflatable warming device on postoperative recovery quality in patients undergoing hysteroscopic myomectomy, this randomized controlled trial enrolled 258 patients receiving laryngeal mask general anesthesia at Xuzhou Central Hospital, China, between March 2022 and August 2024. Patients were allocated to perioperative temperature management with an inflatable warming device set to 38 °C (Group L, n = 129) or 43 °C (Group H, n = 129). A total of 211 patients were included in the final analysis after accounting for exclusions. At 24 h postoperatively, the QoR-40 score was significantly higher in Group H compared to Group L (P < 0.05). At baseline (T0), no significant differences in core body temperature, mean arterial pressure (MAP), or heart rate (HR) were observed between groups (P > 0.05). From T1 to T6, Group H exhibited higher and more stable core body temperature, MAP, and HR compared to Group L (P < 0.05). There were no statistically significant differences in QoR-40 scores at 48 h (P > 0.05) or in the incidence of postoperative restlessness, chills, and infection between the two groups. Compared to 38 °C, the perioperative application of a 43 °C inflatable warming device improved early postoperative recovery quality at 24 h in hysteroscopic myomectomy patients without increasing complication risks.
10 Apr 2026
Read appraisal →JMIR research protocols
Enhancing Continuous Medication Safety Through e-Prescription and Clinical Decision Support Systems in Outpatient Practices and Pharmacies: Protocol for a Multiperspective Study (eRIKA Study)
BACKGROUND: Increased life expectancy is associated with increasing multimorbidity and polypharmacy, leading to a heightened risk of drug-drug interactions and adverse events, especially when multiple health care providers are involved. To address the urgent need for safer medication management in this population, tools such as medication plans (MP), electronic prescriptions (e-prescriptions), and clinical decision support systems (CDSS) offer valuable support. These instruments have the potential to enhance medication safety by providing physicians and pharmacists with a comprehensive overview of a patient's overall medication regimen and by assisting health care professionals in making informed prescribing decisions. OBJECTIVE: This study aims to improve medication therapy safety by combining e-prescriptions, the use of claims data, MPs, CDSS, and interprofessional communication. To comprehensively evaluate this complex intervention, a holistic multiphase study will be conducted, examining (1) the effectiveness of the intervention and (2) health-economic and (3) implementation-related aspects. METHODS: A multiphase study design is used. In the first phase, the intervention is implemented in selected outpatient practices (n=10) and pharmacies (n=10) in 2 regions in Germany as part of a cluster-randomized controlled trial to assess process-related outcomes. The primary outcome is the congruence between the MP and claims data. In phase 2, the intervention is scaled up in 3 regions and evaluated in a quasi-experimental study. The required sample size for the intervention group is 3528 patients, with a synthetic control group matched from existing claims data. The primary outcome is a combined end point of all-cause mortality and hospitalization within 3 months of an index prescription. Quantitative methods (descriptive, regression-based methods using claims data, calculation of the incremental cost-effectiveness ratio, and survey-based analyses of implementation-related aspects) and qualitative methods (interviews and focus groups to capture experiences of health care professionals and patients) are used. RESULTS: In phase 1, a total of 187 patients were recruited (74 in the intervention group and 113 in the control group) by June 2025. Phase 2 is currently ongoing, with data collection continuing through December 31, 2025. Final analyses are planned by March 2027. CONCLUSIONS: Medication safety in polypharmacy remains a critical challenge in Germany. This study provides multiperspective evidence supporting the nationwide implementation of the eRIKA (e-prescription as an element of interprofessional care pathways for continuous medication therapy management [eRezept als Element interprofessioneller Versorgungspfade für kontinuierliche AMTS]) intervention.
8 Apr 2026
Read appraisal →JMIR mental health
It Is the Journey, Not the Destination: Moving From End Points to Trajectories When Assessing Chatbot Mental Health Safety
Large language models are rapidly becoming embedded in everyday life through artificial intelligence (AI) chatbots that people use for practical assistance and companionship, as well as for support with mental health and emotional well-being. Alongside clear benefits, clinicians and public reports increasingly describe a minority of users whose interactions seem to drift over days or weeks toward strongly questionable convictions, delusions, or suicidal crises. Importantly, clinically meaningful deterioration can occur even without overtly unsafe text outputs, via more insidious processes, such as compulsive use, sleep disruption, withdrawal from human contact, and progressive narrowing of attention around the chatbot relationship. In this Viewpoint, we argue that risk often arises not at a single tipping point but through trajectory effects that accumulate across extended dialogue and that prevailing safety evaluation approaches are misaligned with this reality because they primarily score risk at discrete conversational end points often reached through scripted dialogues lasting just a single turn or several turns. Mental health benchmarks and safety suites (including clinician-informed efforts) have advanced the field by testing refusal behavior, toxicity, and adversarial prompting. However, they often treat the last message as the unit of analysis and, therefore, miss when risk-relevant relational cues, signs of validation, contradiction handling, and shifts in certainty first emerge and how they compound. We propose that mental health safety assessment should shift from end points to trajectories by (1) treating the whole dialogue, not just the end result, as the focus of evaluation; (2) reporting turn-by-turn dynamics, such as delusion confirmation and harm enablement, and timing and persistence of safety interventions; and (3) calibrating short multiturn tests against longer, clinically realistic interaction sequences that can reveal context-length effects and drift. We further argue that transcript-only evaluation is insufficient in mental health contexts. Similar language can reflect very different internal states, and the relationship between expressed psychopathology and real-world harm is nonlinear. Therefore, safety research should incorporate proximal human outcomes following interactions (eg, shifts in certainty, openness to counterevidence, arousal, urge to continue, and subsequent sleep or behavior) and build a prospective clinical surveillance infrastructure that supports transcript donation with consent and linkage to health outcomes. Together, these steps would enable benchmarks that are clinically relevant and better aligned with the types of harms now being observed in real-world chatbot use.
7 Apr 2026
Read appraisal →Annals of plastic surgery
Predicting Surgical Outcomes in Breast Reconstruction With Machine Learning: A Systematic Review
INTRODUCTION: The applications of artificial intelligence (AI) in plastic surgery have grown considerably in recent years. As large patient datasets become more accessible, surgeons are increasingly leveraging machine learning (ML), a subset of AI, to predict patient outcomes and guide surgical decision-making. This review evaluates the relative performance of ML prediction models in breast reconstruction. METHODS: A systematic review was conducted utilizing PubMed, Scopus, and EMBASE according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies using ML to predict patient outcomes in breast reconstruction were included. The type of ML model and the specific outcome measures were reported. Performance of the models was reported as area under the receiver operating characteristic curve and compared using descriptive statistics, multivariate linear regression, and random-effects meta-regression in RStudio. RESULTS: Our search yielded 1025 citations, of which 24 were assessed for eligibility. Fourteen studies met the inclusion criteria and were sought for data extraction. There were 19 ML models and 11,013 patients assessed across 92 testing conditions. Models were trained on varying patient demographics, comorbidities, and operative characteristics, whereas outcomes assessed included various surgical complications or patient satisfaction using BREAST-Q. The median area under the receiver operating characteristic curve of all models was 0.71 (interquartile range = 0.16). When adjusting for the number of patients, number of predictors, ML model category, and outcome of interest, models predicting BREAST-Q performed higher with skin necrosis used as the reference outcome (β = 0.13, P < 0.01). After adjusting for number of patients and predictors, models that employed strategies to mitigate class imbalance were associated with higher model discrimination (β = 0.038; 95% CI, 0.002-0.075; P = 0.041). CONCLUSION: Machine learning applications for risk prediction and surgical planning are growing rapidly. The models evaluated in this review demonstrated the ability to predict a variety of outcomes, with models predicting BREAST-Q, various surgical outcomes, and those reporting class imbalance methods leading to higher model discrimination. Notably, covariate adjustment and study heterogeneity may have impacted these associations. As ML models are increasingly integrated into plastic surgery practice, standardized reporting practices are essential to promote reproducibility and cross-study comparison.
4 Apr 2026
Read appraisal →