Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 9 appraisals
Telemedicine journal and e-health : the official journal of the American Telemedicine Association
Telehealth Competency Evaluation Tools: A Scoping Review
INTRODUCTION: Telehealth is an integral part of healthcare. Exposure to telehealth education is essential for both students and health care professionals to support its effective adoption and appropriate utilization. This scoping review aims to explore whether and which telehealth competencies are being assessed and the methods used to evaluate them among health care professionals and students. METHODS: An electronic literature search was performed using six electronic databases between January 2024 and February 2025. We included studies where telehealth competencies or their components were evaluated by some type of tool (validated or researcher created). Strict inclusion and exclusion criteria were used, and each article was evaluated by at least two reviewers. RESULTS: Out of 1,217 articles screened by title and abstract, 75 met inclusion criteria, with 36 of those selected for inclusion in this review. Participants were evaluated mostly by researcher-created tools that lacked psychometric properties or adapted nontelehealth evaluation tools. Few validated/reliable tools directly measured telehealth competencies. DISCUSSION: The results of this review illustrate there exists a lack of available validated/reliable evaluation tools to appraise telehealth competencies. Few studies included an evaluation of all telehealth competencies but rather focused only on communication and technology proficiency; none evaluated the appropriate use of telehealth or digital disparities. CONCLUSIONS: More research is needed to develop validated telehealth evaluation tools that can be used across disciplines and encompass all telehealth competencies. Additionally, true research methods are needed to adequately assess the impact of telehealth education on the performance and confidence of learners.
3 Aug 2026
Read appraisal →Rheumatic diseases clinics of North America
Demystifying Artificial Intelligence: Key Concepts with Examples in Rheumatology
Artificial intelligence (AI) refers to a broad class of computational methods to perform tasks that typically require human intelligence, such as learning patterns, reasoning, and problem solving. AI is increasingly applied across rheumatology research and clinical practice to analyze complex data and support clinical decision-making, yet its underlying concepts are often perceived as opaque or inaccessible. This aim of this article is to demystify foundational AI concepts by defining core terminology and describing major learning paradigms and analytical methods, highlighted through clinically relevant examples from rheumatology. We explain how modern AI models differ from traditional analytical approaches.
2 Aug 2026
Read appraisal →JMIR research protocols
Large Language Models in German Continuing Medical Education Assessments: Protocol for a Fully Crossed Experimental Study
BACKGROUND: Continuing medical education (CME) is a legal and ethical obligation for physicians in Germany. The rapid rise of large language models (LLMs) such as ChatGPT, Gemini, Claude, and Grok raises concerns about the integrity of CME assessments, as LLMs can already pass German CME tests. OBJECTIVE: This study aims to determine whether the choice of document format (searchable PDF, protected PDF, raster PDF, or vector PDF) and LLM influences the ability of LLMs to solve CME test questions at rates exceeding the passing threshold specified for each CME module (typically 70%). METHODS: In a fully crossed within-subjects repeated-measures design, 18 expired CME articles from 3 major German publishers across 6 specialties will be converted into 3 cheating-impeding PDF formats and processed alongside the original PDF files by 4 current LLMs (GPT-5, Claude Sonnet 4, Grok-4, and Gemini 3). This results in 16 model-format combinations. Each model will answer every article 3 times per file-format condition, with outcomes derived from aggregated run-level results. The primary outcome is the proportion of correctly answered questions; the secondary outcome is the pass/fail rate. RESULTS: The study has been approved by the Witten/Herdecke University Ethics Committee (S-260/2025; dated August 10, 2025) and is preregistered at the Open Science Framework. The study is supported by internal departmental resources only, and no external funding was received. Because this protocol evaluates LLMs using expired CME materials, no human participants are being recruited. Data collection is planned to begin in June 2026 and is expected to last approximately 4 weeks. At the time of manuscript submission, no data have been collected or analyzed. Results are expected to be available after the completion of data collection and statistical analysis in 2026. The analyses will quantify performance differences across document formats; these findings may inform the feasibility of nonsearchable document formats as a temporary measure to reduce LLM-enabled cheating risks in CME contexts. CONCLUSIONS: By quantifying how document format constrains LLM performance, this study aims to evaluate simple technical safeguards that may reduce artificial intelligence-assisted manipulation of CME tests and inform regulators and CME providers about how to balance assessment validity, accessibility, and responsible LLM integration into postgraduate medical education.
30 July 2026
Read appraisal →PloS one
"We'll have to see how it works": An interview study to understand collaborative practices in interdisciplinary artificial intelligence and healthcare research.
Developing artificial intelligence (AI) algorithms for healthcare is a collaborative effort, bringing data scientists, clinicians, patients and other stakeholders together. By understanding AI as 'sociotechnical' where the social and the technical nature of the work and the models are inseparable, we explore the AI development workflow and how stakeholders navigate the challenges and tensions of sharing and generating knowledge across disciplines. We conducted an inductive thematic analysis of 13 semi-structured interviews with participants in early stages of AI-in-healthcare research consortia in the UK. Our findings identify that participants needed to adapt both the tools used for sharing and the information communicated according to their audience, particularly when working with those with a clinical or patient perspective. We identify the novelty of participating in AI research, how AI knowledge is shared, and the inclusion of clinician and patient stakeholder perspectives as key areas within collaborative AI practices in healthcare. These findings highlight that bringing AI into the mix can introduce new obstacles to interdisciplinary work.
24 July 2026
Read appraisal →Journal of hospital medicine
Gender differences in secure chat and electronic health records use among hospital-based physicians
BACKGROUND: Digital messaging within electronic health records (EHRs) is central to inpatient communication. While intended to enhance efficiency, these tools may contribute to unequal digital workloads. OBJECTIVE: To evaluate gender-based differences in EHR and secure messaging use among hospital-based physicians. METHODS: Retrospective observational study at a single academic tertiary care hospital using EHR metadata from July 2023 to June 2024. Participants included a total of 205 internal medicine and medicine-pediatrics physicians (108 senior residents, 97 faculty) serving as primary clinicians on hospitalist shifts. Measures included daily number of messages sent/received, time spent in EHR and secure chat, and hours worked, stratified by gender and role. RESULTS: Physicians worked a median of 9.4 h/day, spending 40.7% of their time in the EHR and 6.3% in secure chat. Women and men work similar hours (9.5 vs. 9.3, p = .40) but women spent more time in the EHR (248 vs. 222 min/day; p < .001) and secure chat (38 vs. 34 min, p < .001) and exchanged more daily messages (62 vs. 53 sent; 56 vs. 48 received, both p < .001). Patterns were consistent across residents and faculty. Among faculty, gender differences persisted after a new scheduling model, which reduced overall work hours. CONCLUSIONS: Women physicians engaged more in digital communication than men, despite similar hours worked, and these differences persisted after a workflow change. These findings underscore the need for equity-informed strategies that both mitigate excess burden and recognize the potential value of proactive communication in hospital settings.
3 July 2026
Read appraisal →Journal of the American Medical Informatics Association : JAMIA
Clinicians' rationale for editing ambient AI-drafted clinical notes: persistent challenges and implications for improvement
OBJECTIVE: The use of ambient AI documentation tools is rapidly growing in US hospitals and clinics. Such tools generate the first draft of clinical notes from scribed patient-provider conversations, which clinicians can then review and edit before signing into electronic health records (EHR). Understanding how and why clinicians make modifications to AI-generated drafts is critical to improving AI design and clinical efficiency, yet it has been under-studied. This study aims to address this gap. MATERIALS AND METHODS: We conducted semistructured interviews with 30 clinicians from the University of California, Irvine Health who used a commercial ambient AI tool in routine outpatient care. We invited them to describe how and why they edited AI drafts based on both their personal experience and review of some real-world examples identified from our previous studies. RESULTS: Modifications to AI drafts were primarily made to improve clinical accuracy and specialty-specific precision, reduce medico-legal and liability risk, and meet billing, coding, and documentation standards. Such editing was necessary due to reasons such as transcription errors, speaker attribution mistakes, overconfident statements without evidence, missing key clinical details, and AI's lack of information about the patient context. CONCLUSION AND DISCUSSION: Improving ambient AI documentation will require coordinated effort from vendors, institutions, and clinicians. Key targets include core model reliability (eg, transcription accuracy), specialty- and encounter-level customization, clinician-level personalization, more effective EHR integration, and institutional support (eg, training, governance, and standardized review guidance), complemented by clinicians' adaptive communication strategies that strengthen human-AI collaboration.
2 July 2026
Read appraisal →Journal of the American College of Clinical Pharmacy : JACCP
Research and Scholarly Methods: Using Artificial Intelligence
Artificial intelligence (AI) is quickly changing health care and expanding research opportunities, including in pharmacy. AI can offer effective solutions for numerous challenges in pharmacy research, such as persistent gaps in evidence-based prescribing, slow drug discovery-to-adoption timelines, complex data environments, and the need for more efficient methods, including qualitative methods. AI tools-ranging from machine learning models to large language models (LLMs)-offer new capabilities for data analysis, prediction, and content generation. Pharmacy researchers can harness these technologies to address persistent gaps in medication management, optimize patient safety and outcomes, and streamline many research workflows. However, successful integration requires understanding the spectrum of AI capabilities, appropriate use cases, and potential limitations. AI must be tailored to the research question, data, and institutional resources. This paper will guide pharmacy researchers in identifying use cases for AI in their research, selecting and applying AI tools, mitigating risks, providing standards for reporting the use of AI in research, and maximizing benefits for scientific discovery and patient outcomes.
1 July 2026
Read appraisal →Proceedings of the National Academy of Sciences of the United States of America
Training humans to detect AI-generated faces
As AI-generated faces become indistinguishable from real ones, deepfake technology poses escalating threats to information integrity and security. While algorithms can detect deepfakes, they suffer from opacity and critical vulnerabilities-and training humans to identify specific visual artifacts has proven largely ineffective. Here, we introduce a fundamentally different approach that harnesses people's global impressions of faces. Building on findings that AI and human faces evoke systematically different perceptual impressions (Miller et al., 2023), we trained participants to attend to these distinguishing qualities without explicit instruction on how to use them. Using a rigorous pre-post design with untrained test faces, we demonstrate that all participants (N = 45) improved, with mean accuracy nearly doubling. High performers achieved near-perfect detection, and participants developed metacognitive insight, showing appropriate confidence calibration only after training. A test-retest control study ruled out practice effects as an explanation for training gains, and an online replication demonstrates the scalability of our approach. As the technology advances, targeting systematic biases that are inherent to generative AI, and which manifest in global features, may offer a more durable defense than approaches reliant on image artifacts alone.
30 June 2026
Read appraisal →JMIR nursing
Digital Health Literacy and Attitudes Toward Telehealth Use in Practice Among Nursing Students in Saudi Arabia: Cross-Sectional Study
BACKGROUND: Nursing students are the future workforce, and their readiness to use digital health tools is important. Previous studies have focused on knowledge and attitudes; however, they have not examined the wide range of digital health literacy levels that may influence nursing students' attitudes toward using telehealth in clinical settings. OBJECTIVE: This study aimed to determine the relationship between nursing students' digital health literacy and their attitudes toward telehealth use in practice. METHODS: A cross-sectional design was used. The sample consisted of undergraduate nursing students enrolled in a Bachelor of Nursing program at a selected Saudi Arabian university. The online survey used 2 scales: the Digital Health Care Literacy Scale and the Nurses' Attitudes Toward Use of a Telehealth Scale. RESULTS: A total of 273 students participated (mean age 21.3, SD 1.9 years). Most of the nursing students demonstrated a high digital health literacy level (n=184, 67.4%; mean Digital Health Care Literacy Scale score 11.9 out of 15, SD 3.1). Digital health literacy was a significant predictor of positive attitudes toward telehealth use in practice (adjusted odds ratio 1.48, 95% CI 1.28-1.71; P<.001). Male students were significantly less likely to report positive attitudes than female students (adjusted odds ratio 0.62, 95% CI 0.39-0.97; P=.03). However, academic year, telehealth workshops, and informatics courses were not significantly associated with positive attitudes toward telehealth use in practice. CONCLUSIONS: Higher levels of literacy appear to correlate with more positive attitudes toward telehealth use in practice. However, current formal education and workshops had no apparent influence on digital health literacy. This suggests a potential need for strengthening digital training and development in nursing education. This may enhance telehealth readiness and support future digital health care delivery.
24 June 2026
Read appraisal →