Next-generation multicenter studies: using artificial intelligence to automatically process unstructured health records of patients with lung cancer across multiple institutions

Clinical Snapshot

Key Findings

• P Value: P <0.001 for primary comparison

• Effect Size: 50% relative reduction in error rate with AI extraction

• Primary Outcome: AI extraction error rate 7.0% vs manual abstraction 14.2%

• Nnt Or Sensitivity: F1-scores: gene alterations 0.97 vs 0.86, comorbidities 0.86 vs 0.76, metastatic sites 0.71 vs 0.69

• Confidence Interval: Not explicitly reported for primary endpoint

Clinical Application

Requires institutional investment in AI infrastructure and validation studies; hybrid approach with 30% human review most practical Would require validation within Australian electronic health record systems and compliance with Privacy Act 1988; potential integration with existing cancer registries and ANZICS databases Lung cancer patients in multicenter research settings requiring large-scale data extraction from unstructured health records

Abstract

BACKGROUND: Manual abstraction of real-world data (RWD) from unstructured health records (HRs) remains resource intensive, error prone, and highly variable across institutions. Large language models (LLMs) offer a scalable alternative, but their performance in multicenter oncology settings is not fully validated. PATIENTS AND METHODS: We conducted a multicenter study within the French Large & Unified Cancer Cohort (LUCC) consortium to compare the accuracy of artificial intelligence (AI)-based data extraction against manual abstraction by clinical research professionals. A fine-tuned LLM was applied to de-identified unstructured HRs in PDF format to extract 31 variables from lung cancer patients across 10 centers. Ground truth was defined as concordant values across sources, with discrepant cases adjudicated by a blinded expert. The primary endpoint was the extraction error rates. Secondary endpoints included per-variable performance, interinstitutional variability, F1-score for multiple-choice variables, added value of hybrid AI-human workflows, and survival analyses. RESULTS: Among 10 327 patients with AI-based extraction, 311 were included in the test cohort. Across 8708 datapoints for 28 variables with only one correct answer, the LLM achieved a 7.0% error rate, outperforming manual abstraction (14.2%, P <0.001). The F1-scores of three multiple-choice variables were superior (gene alterations 0.97 versus 0.86, comorbidities 0.86 versus 0.76, metastatic sites 0.71 versus 0.69). Interinstitutional variance was lower with AI (0.12% versus 0.39%). A hybrid approach with targeted human review of 30% of low-confidence AI outputs further decreased error rates to 4.4%. Survival analyses based on AI-extracted data closely matched ground truth, with similar median overall and progression-free survival. CONCLUSIONS: In a multicenter setting, our AI pipeline yielded lower error rates and greater consistency than manual abstraction. These findings support the feasibility of next-generation, AI-enabled multicenter studies to generate high-quality RWD at scale, with potential applicability in prospective clinical trials.

References

1. 1.Aldea, M., Zullo, L., Levrat, V., Bennouna, J., Schneider, S., Mercier, O., Mougenot, E., Bergot, E., Dujon, C., Cloarec, N., Audigier Valette, C., Nuccio, A., Deloger, M., Helissey, C., Simon, S., Carpentier, A., Djarallah, A., Rolland, P., Louis, J. C., Ancillon, L., Vignal, B., Rambaud, F., Tessier, P., Chuttoo, L., Siby, K., Poplu, A., Zarca, K., Michiels, S., Barlesi, F., Le Ouay, F., & Besse, B. (2026). Next-generation multicenter studies: using artificial intelligence to automatically process unstructured health records of patients with lung cancer across multiple institutions. Annals of Oncology, Article 10.1016/j.annonc.2025.12.006. https://doi.org/10.1016/j.annonc.2025.12.006