Automated data extraction from electronic medical records for pragmatic clinical trials

Clinical Snapshot

Key Findings

• P Value: p < 0.001 for intraclass correlation coefficients

• Effect Size: ADE correct in 77.9% of disagreements vs MDE 22.1%

• Primary Outcome: Agreement between automated and manual data extraction methods

• Nnt Or Sensitivity: Overall accuracy: 99.64% (inaccurate data points 0.36% of total)

• Confidence Interval: Not reported for primary comparison

Clinical Application

High feasibility for structured EMR data but requires significant upfront investment in SQL programming and validation Relevant for Australian ICUs using electronic medical records, though EMR system differences may affect implementation. Could support TGA-required data collection for pragmatic trials and align with NHMRC priorities for efficient trial conduct ICU patients in pragmatic clinical trials requiring comprehensive data extraction from electronic medical records

Abstract

Data collection in randomized trials is expensive and labor intensive. With the rise in ongoing pragmatic trials, the use of electronic medical records (EMR) as a source of data has increased. Although potentially faster and cheaper, EMR use can lead to errors. Therefore, to ensure accurate data collection and to avoid systematic errors we performed a study comparing automated data extraction (ADE) with manual data extraction (MDE). We performed a retrospective cohort study to compare the accuracy of ADE using Structured Query Language with MDE by blinded physicians from our EMR. We tested the interrater agreement and intraclass correlation coefficient of clinical baseline data and outcomes of a random sample of 30 patients admitted to the ICU, on mechanical ventilation, requiring opioids for analgosedation for an upcoming pragmatic clinical trial. Key data compared included, but not limited to, patient's demographics, laboratory and vital signs, daily morphine milligram equivalent (MME), days alive and free of mechanical ventilation, days alive and free of hospitalization, days alive and free of ICU, days alive and free of vasopressors, and death. Among 238 patients screened over 1-month period, 72 fulfilled inclusion criteria and 30 were randomly selected to be included in the evaluation. We blindly collected 1320 baseline data, 2160 categorical outcomes and 705 continuous outcomes for a total of 4185 data points. The intraclass correlation coefficient and the Cohen's Kappa were perfect or almost perfect for all data, including outcomes such as daily MME, days alive and free of mechanical ventilation, days alive and free of ICU and days alive and free of hospital with p < 0.001. Among all rechecked data, the ADE was correct in 53 (77.9%) of cases, while MDE in 15 (22.1%). The inaccurate data collected by ADE accounted for 0.36% of the total data-points. The performance of ADE had almost perfect agreement for all outcomes and when rechecking for disagreements, it was more accurate than MDE.

References

1. 1.Padrao, E. M. H., Nguyen, A. T., Nguyen, K., Sopuch, I. A., Gulluoglu, A., Alape, M. A., Harrison, S., Wong, A., Sadrolashrafi, M., Cozzi, G., Pais, K., Furfaro, D., Hayes, M., Goodspeed, V., Talmor, D. S., & Baedorf-Kassis, E. N. (2026). Automated data extraction from electronic medical records for pragmatic clinical trials. Journal of Medical Systems. https://doi.org/10.1097/MAT.0000000000001253