Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 6 appraisals
International journal of medical informatics
Human in the loop artificial intelligence in healthcare: applications, outcomes, and implementation challenges
BACKGROUND: The integration of artificial intelligence in healthcare has transformed clinical practice and research methodologies. However, concerns regarding algorithmic accountability, interpretability, and safety have necessitated human oversight in AI systems. Human in the loop artificial intelligence represents a collaborative paradigm where human expertise and machine intelligence converge to enhance decision making while maintaining ethical standards and clinical safety. AIM: This review synthesizes current evidence on human in the loop AI in healthcare delivery and research, examining implementation frameworks, clinical outcomes, comparative advantages over fully automated and clinician-only approaches, and challenges. METHOD: A comprehensive narrative review was conducted using PubMed, Scopus, Web of Science, and IEEE Xplore databases covering studies from 2018 to 2025. Data were thematically synthesized to identify patterns, frameworks, and outcomes. This narrative approach enables comprehensive conceptual synthesis across diverse HITL-AI applications and contexts. RESULTS: Human in the loop AI demonstrates significant applications across diagnostic imaging, clinical decision support, patient monitoring, drug discovery, and research data analysis. Evidence indicates improved diagnostic accuracy, reduced medical errors, enhanced patient safety, and increased clinician trust compared to both automated AI and traditional approaches. Implementation requires EHR interoperability, clear liability frameworks, adaptive training protocols, and quantum-safe cryptographic security. Challenges include workflow integration, regulatory gaps for adaptive systems, and sustainability concerns. CONCLUSION: This review advances the field by synthesizing cross-domain implementation patterns, mapping collaboration models to risk-stratified contexts, identifying regulatory gaps for adaptive systems, and proposing future directions including post-quantum cryptographic integration, AI-driven adaptive architectures, and multi-center scalability frameworks for optimizing human-machine collaboration in healthcare.
15 June 2026
Read appraisal →Journal of medical Internet research
Co-Lifecycle Governance for Learning Medical AI: A Hybrid Convergence Framework for Adaptive Regulatory Oversight
Artificial intelligence (AI) in health care is increasingly defined not by static algorithms but by adaptive intelligence-systems that evolve over time through interactions with data, clinicians, and clinical environments. This adaptive capacity creates a structural mismatch with regulatory frameworks built for technologies whose behavior remains static. As AI models drift, recalibrate, or degrade in real-world contexts, they dissolve the linear boundaries between design, deployment, and clinical interpretation. These temporal, epistemic, and organizational frictions expose responsibility gaps that cannot be resolved through incremental modifications to legacy oversight structures. Regulators across major jurisdictions are beginning to respond to these challenges, though with differing orientations. The United States advances mechanisms for predictable adaptation, including Predetermined Change Control Plans, real-world evidence frameworks, and life cycle-oriented quality management reforms. The European Union emphasizes precautionary, rights-based governance through the European Union Artificial Intelligence Act (AI Act) and modernized liability rules. South Korea, operating within a hyperconnected digital health ecosystem, has introduced the Digital Medical Products Act (DMPA), one of the world's first comprehensive statutory frameworks for learning medical AI. Despite philosophical differences, these regulatory trajectories converge on a shared insight: learning AI systems cannot be governed by static rules or episodic evaluation. This viewpoint proposes Co-Lifecycle Governance as a conceptual framework to synchronize regulatory oversight with adaptive intelligence. Rather than treating oversight as a discrete event, Co-Lifecycle Governance frames regulation as a continuous, synchronized process grounded in 4 pillars: continuous validation, agile change management, proactive performance surveillance, and distributed accountability. Each pillar functions as a structural antidote to the responsibility frictions that arise when AI systems evolve faster than expectations surrounding them. Together, these pillars provide a governance grammar capable of supporting safe, iterative model improvement while maintaining system-level trust. Drawing from the strengths of US predictability, European Union accountability, and Korean scalability, this paper outlines a hybrid convergence pathway that synthesizes predictability, accountability, and operational feasibility. Learning AI will not wait for governance to catch up; oversight must evolve in lockstep with adaptive intelligence. Co-Lifecycle Governance offers a foundation for regulatory systems that not only regulate learning AI but also learn with it-at the speed at which adaptive intelligence actually changes.
20 May 2026
Read appraisal →JMIR human factors
Predicting Electronic Health Record Usability: Scoping Review of Adoption Models, Metrics, and Future Directions
BACKGROUND: Electronic health records (EHRs) play an essential role in modern health care, enabling data sharing and improving patient safety; however, even though vendors must adhere to International Organization for Standardization-related usability standards for EHR certification, persistent usability issues continue to undermine efficiency, contribute to clinician burden, and increase the risk of preventable errors. OBJECTIVE: This scoping review synthesizes existing research on EHR adoption and usability, emphasizing theoretical models, measurement approaches, factors, and analytic methods used to assess or predict usability. We identify gaps and opportunities for integrating predictive analytics and artificial intelligence (AI) to advance research and improve the usability of EHRs. METHODS: Following Joanna Briggs Institute and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, we systematically searched MEDLINE, Web of Science, IEEE Xplore, and Scopus library databases for studies published between January 1, 2009, and April 9, 2025. Inclusion criteria focused on empirical research using predictive methods or models related to EHR usability. Data were charted and synthesized thematically. RESULTS: Of the 2323 screened papers, 47 studies met inclusion criteria. Most research examined or predicted EHR adoption (not usability) using dominant frameworks, such as the technology acceptance model, unified theory of acceptance and use of technology, and the information system success model, which comprised usability. Factors related to usability-particularly perceived usefulness, perceived ease of use, effort expectancy, and facilitating conditions via the EHR adoption models-appeared frequently. Regression-based methods and structural equation modeling were the most common analytic techniques. No studies applied predictive modeling or AI to predict EHR usability. CONCLUSIONS: The focus of this study on the prediction of EHR usability and adoption for the past 15 years is distinctive in the literature. It extends prior usability reviews (mostly focusing on adoption, not prediction of usability). Predictive modeling for EHR usability remains underdeveloped throughout 2009 to 2025. Dominant frameworks in the EHR literature continue to prioritize predicting adoption over operational usability. These models rely heavily on self-reported, cross-sectional measures captured at a single postimplementation time point, embedding systematic bias and obscuring longitudinal usability dynamics. Despite the application of increasingly sophisticated predictive techniques-primarily variants of regression and structural equation modeling-usability has remained analytically subordinate to adoption and acceptance constructs for more than 15 years. As a result, widely used models, such as the technology acceptance model and unified theory of acceptance and use of technology, position usability merely as an antecedent to intention or use, rather than as an independent, system-level property that can be empirically measured, modeled, and predicted. Therefore, there is substantial opportunity to integrate predictive analytics, AI, and longitudinal usability measures to build dynamic models.
10 Apr 2026
Read appraisal →BMJ open
Facilitators of and barriers to teamwork between operating room nurses and other practitioners: a meta-synthesis of qualitative studies
OBJECTIVES: Seamless collaboration in the operating room is the critical foundation for patient safety and successful outcomes. This synergy enhances efficiency, minimises errors and is a key determinant of the final surgical quality. The objective of this study is to synthesise qualitative studies on facilitators of and barriers to teamwork between operating room nurses (ORNs) and other practitioners. DESIGN: Systematic review and meta-synthesis of qualitative studies. DATA SOURCES: A systematic search was conducted across nine electronic databases: PubMed, Embase, Scopus, Web of Science (Core Collection), CINAHL, PsycINFO, CNKI, WanFang and Vip. The search covered the period from 1 January 2014 to 30 September 2025. The search strategy used both medical subject headings and free-text terms, supplemented by the snowball method and grey literature. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Qualitative studies exploring teamwork involving ORNs were included. Of 4839 initially retrieved records, 29 studies finally met the inclusion criteria. DATA EXTRACTION AND SYNTHESIS: Two researchers independently extracted data, including the author, year of publication, country, study design, study objective, interview location, number of participants, age, methods of data collection and analysis and duration of interviews. The key findings were synthesised into two overarching themes: facilitators (eg, professional skills, role awareness, team familiarity, patient-centred culture, effective leadership) of and barriers (eg, individualistic behaviour, hierarchical culture, insufficient resources, punitive management, environmental constraints) to teamwork between ORNs and other practitioners. A meta-synthesis was conducted to synthesise study results from different groups and countries. NVivo 15 was used for data management. RESULTS: A total of 29 qualitative studies (published between 2014 and 2025) were included in this study. Key facilitators of teamwork included professional skills, role awareness, team familiarity, patient-centred culture and effective leadership. Major barriers encompassed individualistic behaviours, punitive management, hierarchical culture, environmental constraints and insufficient resources. These factors worked across individual, interpersonal and systemic levels to shape OR teamwork. CONCLUSION: Optimising OR teamwork requires a multifaceted approach simultaneously addressing these facilitators and barriers. Interventions should simultaneously cultivate individual competencies (eg, skills, role awareness), foster positive team dynamics (eg, familiarity, psychological safety, inclusive leadership), and rectify systemic issues (eg, hierarchical culture, resource constraints, punitive management). In this way, teamwork between ORNs and other practitioners can be enhanced, thereby contributing to improved perioperative safety and outcomes. PROSPERO REGISTRATION NUMBER: CRD42024506414.
9 Apr 2026
Read appraisal →Progress in biomedical engineering (Bristol, England)
Trustworthy AI in digital health: a comprehensive review of robustness and explainability
Ensuring trust in artificial intelligence (AI) systems is essential for the safe and ethical integration of machine learning systems into high-stakes domains such as digital health. Key dimensions, including robustness, explainability, fairness, accountability, and privacy, need to be addressed throughout the AI lifecycle, from problem formulation and data collection to model deployment and human interaction. While various contributions address different aspects of trustworthy AI, a focused synthesis on robustness and explainability, especially tailored to the healthcare context, remains limited. This review addresses that need by organizing recent advancements into an accessible framework, highlighting both technical and practical considerations. We present a structured overview of methods, challenges, and solutions, aiming to support researchers and practitioners in developing reliable and explainable AI (XAI) solutions for digital health. This review article is organized into three main parts. First, we introduce core pillars of trustworthy AI and discuss the technical and ethical challenges they pose, particularly in the context of digital health. Second, we explore application-specific trust considerations across domains such as intensive care, mental health, metabolic disease, and public health surveillance, highlighting how explainability, clinical validation, and human oversight support trust. Lastly, we present recent advancements in techniques aimed at improving robustness under data scarcity and distributional shifts, as well as XAI methods ranging from feature attribution to gradient-based interpretations and counterfactual explanations. This paper is further enriched with detailed discussions of the contributions toward robustness and explainability in digital health, the development of trustworthy AI systems in the era of large language models, and various evaluation metrics for measuring trust and related parameters such as validity, fidelity, and diversity, offering a roadmap for building safer and more reliable AI systems.
8 Apr 2026
Read appraisal →Acad Pediatr
Caregiver and Pediatric Clinician Perspectives on Artificial Intelligence for Language Services
OBJECTIVE: Children and caregivers who use languages other than English (LOE) for pediatric health care experience inequities and poorer overall health. Advancement of large language models has raised questions about the future use of artificial intelligence (AI) for language access. This study explored the perspectives of caregivers who use LOE and pediatric clinicians on using AI to address unmet needs in health care translation and interpretation. METHODS: We conducted semistructured individual interviews with caregivers of pediatric patients who use LOE and pediatric clinicians about the use of AI language technologies in health care. A phone interpreter or fluent team member was used for LOE interviews. Recordings were transcribed, translated as needed, coded, and analyzed using inductive thematic analysis. RESULTS: Twenty caregivers using 11 different LOE and 22 clinicians participated. Themes were consistent across participants, though clinicians were more familiar with the concept of AI. Participants reported use of technologies (eg, Google Translate) for written communication and situations where verbal interpretation was perceived to be inadequate. They were concerned about AI accuracy, privacy, and loss of empathy, but hoped it could provide real-time document translation and more convenient verbal communication. Ease of use, validation, and equitable creation and access were critical for use. CONCLUSIONS: Caregivers and clinicians were open to using AI to fill gaps in translation and interpretation; however, robust validation of AI technology and complementary use with human interpreters and translators is needed. Future research, practice, and policy should focus on integrating AI while investing in human translation and interpretation.
1 Jan 2025
Read appraisal →