Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 2 appraisals
Scientific reports
AIDx: a locally deployable AI system for physician clinical decision support
The dynamic environment of medicine, particularly in settings such as the Emergency Department, challenges physicians with an influx of patient data and the need for swift diagnosis and treatment under high-pressure conditions. Numerous AI models have been developed for medical applications, yet their integration into hospital systems remains limited because they often do not align with existing clinical workflows. To address these challenges, I developed AIDx, an AI-powered system designed to help physicians streamline clinical decision making, improve diagnostic support, and provide an integrated platform for AI-assisted analysis. Unlike many standalone systems, AIDx is designed for interoperability with electronic health record (EHR) systems and can leverage supplementary tools such as updated medical knowledge bases and auxiliary AI models. At its core, AIDx includes AIDx-Copilot, a large language model fine-tuned on de-identified EHR data and optionally grounded using retrieval-augmented generation (RAG) from open medical references. This study is a text-only benchmark assessment. I evaluated AIDx-Copilot using the MultiMedQA benchmark suite under a unified, deterministic, single-pass protocol (temperature=0; no voting). Across nine MultiMedQA subsets (MedQA, PubMedQA, MedMCQA, and medically focused MMLU subjects), AIDx-Copilot achieves a mean accuracy of 83.61% (SD 7.37). I report per-dataset Wilson 95% confidence intervals to quantify uncertainty from finite test set sizes. To isolate component contributions, I conducted ablation experiments comparing the base model, the fine-tuned model without retrieval, and the fine-tuned model with RAG enabled. EHR-based fine-tuning accounts for the majority of the performance gain (+17.8 percentage points over the base model on average), while RAG provides a modest additional benefit (+0.4 points on average) that varies by dataset. A qualitative error analysis of 200 incorrectly answered items identifies knowledge gaps (41.0%) and reasoning errors (38.0%) as the dominant failure modes. Comparative numbers for larger proprietary systems are provided only as context in the supplementary material because they may use different prompts and inference settings. The deployment configuration (quantized weights and on-premises serving) supports fast inference and local deployment. Under the quantized configuration, the system achieves a median latency of 0.84 seconds per query and fits within 28.1 GB of VRAM across two commodity GPUs. The primary results are based on public benchmark evaluations without RAG; I did not perform clinical, user-study, or real-world validations. In addition, while AIDx is designed to support fully local operation, specific deployments may choose to use third-party embedding or vector-search services; this paper documents the data-flow boundary and a fully local configuration path.
14 June 2026
Read appraisal →JMIR formative research
MyLungHealth, a Patient-Facing Education Tool for Lung Cancer Screening: Qualitative User-Centered Design Study
BACKGROUND: Lung cancer remains the leading cause of cancer-related mortality worldwide, with low-dose computed tomography screening demonstrating an approximately 20% reduction in mortality among high-risk individuals. Despite this benefit, screening prevalence remains suboptimal, with often less than 20% of eligible individuals reported to be up to date on screening. Shared decision-making is essential for effective lung cancer screening (LCS) implementation, with decision aids shown to enhance patient knowledge and engagement. OBJECTIVE: The aim of this study is to identify patient preferences, concerns, and design considerations through qualitative evaluation of MyLungHealth, a personalized patient-facing educational tool for LCS integrated with electronic health records, and to describe how these findings informed iterative design modifications. METHODS: We employed qualitative research methods through focus groups (n=34) and individual interviews (n=18) with individuals who met screening eligibility criteria. Participants were recruited from the University of Utah Health and New York University Langone Health between May and December 2023. Feedback was analyzed using Braun and Clarke's thematic analysis principles. RESULTS: Six themes were organized into three overarching domains. Domain A included interpretation and impact of personalized risk information: theme 1, difficulties interpreting risk information, and theme 2, varied impacts of risk information on motivation. Domain B included autonomy, privacy, and user interface preferences: theme 3, desire for autonomy and control over personal health data, and theme 4, preference for straightforward language and multiple information formats. Domain C included integration with clinical workflows and patient portal systems: theme 5, expectations for integration with health care provider workflows, and theme 6, mixed experiences with personal health record systems. These insights led to key design modifications, including simplified risk presentation, multimodal content delivery options (video and text), and implementation of electronic health record alerts for clinicians. CONCLUSIONS: The user-centered design process for MyLungHealth revealed important considerations for developing effective patient education tools for LCS. The findings highlighted the need for simplified risk presentation, personalized information delivery, and integration with clinical workflows. These findings underscore the importance of balancing comprehensive risk communication with user accessibility.
23 May 2026
Read appraisal →