Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 6 appraisals
JMIR medical informatics
Extracting Social Determinants of Health From Electronic Health Records: Development and Comparison of Rule-Based and Large Language Model Methods
BACKGROUND: Social determinants of health (SDoH) are critical drivers of health outcomes but are often underdocumented in structured electronic health record (EHR) data. Instead, SDoH are more commonly recorded in unstructured clinical notes, and unlocking this information could have far-reaching implications for advancing population health research and informing clinical decision-making. OBJECTIVE: This study develops and systematically evaluates cost-efficient methods for extracting SDoH information from unstructured clinical text using rule-based natural language processing (NLP) and large language model (LLM)-based approaches. METHODS: We constructed a gold-standard annotated corpus comprising clinical text segments from 171 patients in the Mass General Brigham Research Patient Data Registry, covering 7 SDoH domain categories and 23 subcategories. A rule-based system (RBS) was developed and evaluated alongside 7 OpenAI GPT models (GPT-4o, 4.1, 4.1-mini, o4-mini, GPT-5, GPT-5-mini, and o3) under zero-shot and few-shot settings using multiple prompting strategies. We additionally implemented late-fusion ensemble approaches that combined outputs from rule- and LLM-based methods. Performance was assessed using precision, recall, and F1-score, alongside qualitative error analysis. RESULTS: The RBS achieved high precision for SDoH domain categories (0.96) but substantially lower recall (0.68). GPT-based models consistently outperformed the RBS in overall recall and F1-scores. The best domain-level performance was observed for GPT-5 and GPT-5-mini in few-shot settings (F1-score=0.89), while o4-mini achieved the highest subcategory-level performance (F1-score=0.88). A late-fusion ensemble integrating RBS and GPT outputs further improved domain-level performance (F1-score=0.92), with balanced precision (0.93) and recall (0.90), but did not improve subcategory-level performance. CONCLUSIONS: Recent GPT models with advanced reasoning capabilities, including the newly released mini models (eg, o4-mini and GPT-5-mini), demonstrated strong performance for SDoH extraction without task-specific fine-tuning and consistently outperformed the rule-based NLP system. Integrating rule- and LLM-based methods via late fusion further enhanced domain-level extraction performance. Our results demonstrate a cost-efficient framework for the accurate identification of SDoH from clinical text, facilitating downstream population health research and clinical informatics applications.
20 May 2026
Read appraisal →Journal of medical Internet research
Understanding Remission of Long-Term Conditions Through Electronic Health Records: Scoping Review
BACKGROUND: Multiple long-term conditions (MLTCs) require complex and prolonged treatment regimens. Remission in long-term conditions (LTCs) is important for understanding disease progression and evaluating treatment effectiveness. Electronic health records (EHRs) are increasingly used to monitor clinical outcomes, but how remission is defined within EHRs remains unclear. OBJECTIVE: This study aimed to summarize and collate the previous literature on how remission of LTCs has been defined in EHRs. METHODS: Systematic electronic searches were performed on OVID MEDLINE, Embase, CINAHL EBSCO, the Cochrane Library, and the Bielefeld Academic Search Engine for eligible studies published from inception to November 27, 2025. Quantitative studies, published in any language, on adult populations, and using EHRs to assess remission of LTCs, were eligible for inclusion. Studies that did not clearly define remission and studies on cancer remission were excluded. Data were extracted from each eligible study using a structured table. Risk of bias was not assessed, in line with scoping review methodology. A narrative approach was taken to summarize and present data from the included studies. The number and characteristics of studies were described, both overall and by condition. Findings were discussed with clinicians and data experts to ensure applicability in clinical practice. RESULTS: Ninety-one studies were included. Sample sizes ranged from 12 to 72.9 million adults. Studies were conducted in 18 countries, with the majority being from the United States. The majority of included studies used a cohort study design. Studies assessed how remission was defined in 12 LTCs, including inflammatory bowel disease (41/91, 45.1%), type 2 diabetes (n=15, 16.5%), depression (n=15, 16.5%), alcohol or drug misuse (n=8, 8.8%), asthma (n=3, 3.3%), multiple sclerosis (n=3, 3.3%), epilepsy (n=1, 1.1%), anemia (n=1, 1.1%), chronic kidney disease (n=1, 1.1%), autoimmune pancreatitis (n=1, 1.1%), hypertension (n=1, 1.1%), heart failure (n=1, 1.1%), and MLTC (n=1, 1.1%). Remission was typically defined using a combination of clinical codes (n=7, 7.7%), validated rating scales (n=56, 61.5%), biochemical markers (n=29, 31.9%), absence of symptoms (n=10, 11%), absence of condition-specific events (eg, hospital admissions; n=4, 4.4%), and cessation of pharmacological treatments (n=26, 28.6%). There was substantial variation in the criteria and duration of follow-up used to define remission across studies. CONCLUSIONS: This review demonstrates that remission of LTCs can be identified and operationalized within EHRs, although remission criteria varied across studies. The review extends the literature on remission in EHRs by combining evidence synthesis and consultation with clinical and data experts to propose standardized comprehensive definitions to reliably define and implement remission of multiple LTCs in EHR-based research. This will allow cross-study comparisons and present an opportunity to advance understanding of disease trajectories and improve evaluation and monitoring of patient outcomes. Further research may apply, compare, and evaluate standardized definitions across different data sources to assess generalizability and further improve our understanding of remission of LTCs.
20 May 2026
Read appraisal →PloS one
Healthy living environments create healthy physical activity-related habits (ELEVATE): A protocol for a smartphone-based mixed-method study
Physical inactivity remains a key risk factor for adverse public health outcomes, disproportionately burdening lower socioeconomic groups. While individual-level interventions may temporarily increase walking-related physical activity (WPA) for some, they rarely achieve population-wide benefits. In contrast, built and natural environments are better positioned to support sustained WPA at the population level. Despite the importance of the environment to WPA, state-of-the-art research has limitations: it often relies on cross-sectional self-reports, uses coarse, simplistic environmental indicators, and focuses narrowly on residential neighborhoods, overlooking the diverse streetscapes people encounter in daily mobility. Using a transdisciplinary approach, ELEVATE addresses these gaps by 1) assessing how adults perceive and experience the built and natural streetscape environment during their WPA, 2) longitudinally assessing the relationships between streetscape environments and WPA, and 3) developing an artificial intelligence-powered system to suggest place-based environmental interventions that promote WPA. ELEVATE will employ a mixed-methods study design grounded in participatory co-creation, on-site and walk-along interviews to capture the everyday practices and experiences of WPA. We will use deep learning to extract objective environmental features and perceived streetscape qualities from street view images. A sample of approximately 200 adults residing in Amsterdam, The Netherlands, will be recruited. To examine relationships between micro-scale streetscape features and WPA, we will collect seven days of intensive longitudinal data using smartphones, including global positioning system (GPS)-based mobility and step counts. We will use spatiotemporal clustering to identify where people are most and least physically active, and mixed models to assess associations between these environmental measures and step counts. Qualitative and quantitative results will be triangulated in co-created focus groups. Finally, to support evidence-based public health and urban planning interventions, we will develop an AI-powered recommender system that generates context-sensitive (re)design guidelines framing the built and natural environment as a modifiable determinant of WPA. This study will generate evidence on how micro-scale built and natural street environments encountered during daily mobility are associated with WPA in adults. The findings are expected to inform place-based public health strategies by identifying modifiable environmental characteristics associated with physical activity and socioeconomic inequalities in urban settings.
13 May 2026
Read appraisal →Annals of epidemiology
Population-based clustering of co-occurring social determinants: An application of unsupervised machine learning
PURPOSE: This study aimed to develop a cluster-based measure of multiple co-occurring social determinants of health by applying unsupervised machine learning to a population-based cohort, offering a data-driven approach to organize complex social exposures. METHODS: Unsupervised clustering was applied to a population-based cohort of Ontario respondents to six-cycles of the Canadian Community Health Survey (2001-2012) linked to the Canadian census and vital statistics data. Clusters were evaluated using internal metrics, visualization techniques, descriptive analysis and theoretical considerations to determine the optimal number of clusters. Sensitivity analyses were integrated across the iterative clustering process. Premature mortality rates were generated assess validity. RESULTS: Optimal clustering solutions included 4-clusters and 6-clusters. Both cluster solutions revealed distinct social typologies. The 6-cluster solution offered greater granularity and theoretical interpretability. The 4-cluster solution showed greater heterogeneity within certain marginalized groups. Premature mortality rates differed meaningfully across clusters, supporting the clustering approach in capturing risk associated with social exposure. CONCLUSIONS: Unsupervised machine learning methods identified meaningful population subgroups reflecting complex patterns of social exposures. This approach offers a flexible, data-driven method for characterizing social exposures that can be considered alongside theoretical frameworks and used for equity monitoring, intervention planning and policy development.
6 May 2026
Read appraisal →Proceedings of the National Academy of Sciences of the United States of America
The persistence and loss of hard selective sweeps amid admixture in ancient Eurasians
The extent to which human adaptations have persisted throughout history despite strong eroding demographic events such as admixture, genetic drift, and fluctuations in selection pressures remains unknown. Understanding which adaptations were resilient to such forces may shed light on traits that were important for humans across time. Yet, detecting selection from ancient DNA is challenging due to severe degradation of the data and/or signal. Here we detect selective sweeps using a domain-adaptive neural network (DANN) trained on simulated data and applied to more than 800 ancient and modern Eurasian genomes spanning the last 7,000 y. We show that the DANN can account for simulation misspecification, or discrepancies between simulations and real ancient DNA, improving the ability to detect sweeps in real data compared to standard convolutional neural networks or standard statistics. Application of the DANN to data recovered 16 known sweeps at loci including LCT, HLA, KITLG, and OCA2/HERC2, and revealed 32 novel sweeps. All identified sweeps were classified as hard, consistent with historically low population sizes. While some sweeps were lost over time, 14 sweeps at loci involved in functions including neuronal, reproductive, pigmentation, and signaling traits persisted from the earliest to the most recent time periods. In most cases, the most frequent haplotype remained at high frequency across time. Together, these results indicate that hard sweeps predominated in ancient Eurasians and that several ancient selective events were resilient to strong admixture events.
2 May 2026
Read appraisal →JMIR medical informatics
Modeling Diabetes Risk and Progression With Public Health Data: Ontology-Guided, Simulation-Capable Digital Twin Study
BACKGROUND: Digital twins (DTs) offer a paradigm for health care by enabling data-driven, simulation-capable representations of individual health trajectories. However, DT development remains limited by the scarcity of standardized, temporally structured, and multidomain data suitable for modeling chronic disease progression. Most existing DT studies rely on narrowly scoped or proprietary datasets, restricting generalizability. Public health datasets, such as the Midlife in the United States study, provide rich biopsychosocial information but are underused due to structural complexity and lack of semantic integration frameworks. OBJECTIVE: This study aimed to develop and evaluate an ontology-guided, agent-orchestrated framework for constructing offline, simulation-capable, and progression-aware DTs from public health datasets. Using diabetes as a case study, the framework integrates agent-based orchestration, medical ontologies, and large language model (LLM)-assisted semantic reasoning with machine learning to support explainable feature structuring, risk prediction, and predictive "what-if" progression analysis. METHODS: A 6-stage DT framework was developed and applied to Midlife in the United States wave 2 (baseline) and wave 3 (follow-up) data. Ontology- and LLM-assisted feature selection identified predictors across biological, behavioral, psychosocial, and socioeconomic domains. Cleaned and harmonized data were used to train predictive models (random forest, eXtreme gradient boosting, and logistic regression) to estimate diabetes onset at follow-up. A state-transition simulator was implemented to model between-wave progression dynamics, quantify transitions across low-, medium-, and high-risk states, and evaluate predictive "what-if" scenarios such as weight reduction and lifestyle improvement. Model performance was assessed using accuracy, F1 score, area under the receiver operating characteristic curve (AUC), and calibration metrics. RESULTS: From 9976 candidate variables, ontology- and LLM-guided selection retained the top 200 relevant predictors spanning biological, behavioral, psychosocial, and socioeconomic domains. Predictive modeling achieved strong discrimination, with random forest (AUC=0.82, accuracy=0.76) and eXtreme gradient boosting (AUC=0.81, accuracy=0.75) outperforming logistic regression (AUC=0.78). The state-transition simulator reproduced realistic progression patterns: 33.9% (1414/4174) of participants changed risk states between waves, and the high-risk group increased from 10.8% (451/4174) to 32.2% (1344/4174). Next-state prediction accuracy reached 92.5%. Predictive "what-if" analyses showed that with a simulated 10% weight reduction, model-estimated diabetes cases decreased by 98 (from 576 to 478). A placebo test (0% weight change) produced less than 0.3% difference in risk distribution, confirming model stability. CONCLUSIONS: This study presents a foundational, ontology-guided, and agent-orchestrated framework for constructing offline, simulation-capable, and progression-aware DTs from public datasets. By combining semantic reasoning, multidomain predictors, and predictive "what-if" progression simulation, the framework transforms static population data into longitudinal, interpretable representations of individual health trajectories. The proof-of-concept application to diabetes demonstrates that public health data can support robust and explainable DT models for exploratory risk analysis and hypothesis generation, without implying causal intervention effects or direct clinical decision support.
24 Apr 2026
Read appraisal →