Automated Flow and local LLM-Driven clinical Context Engineering: Precision colorectal cancer recurrence registry

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Rule-Based LLM: 90.7% accuracy vs Manual TCR: 92.0% accuracy

• Primary Outcome: Accuracy of automated colorectal cancer recurrence detection

• Nnt Or Sensitivity: Sensitivity: 77.2%, Specificity: 93.9% for Rule-Based LLM

• Confidence Interval: Not reported

Clinical Application

Requires significant technical infrastructure for local LLM deployment and integration with existing electronic health records Potential application in Australian cancer registries and major cancer centres, though would require validation against Australian clinical documentation standards and integration with existing cancer surveillance systems Colorectal cancer patients requiring systematic recurrence surveillance in large healthcare institutions

Abstract

OBJECTIVE: This study develops and validates a deployable, privacy-preserving automated tool to address the labor-intensive nature of colorectal cancer (CRC) recurrence registration. Our tool utilizes a reproducible, two-stage 'Clinical Context Engineering' workflow to mimic expert clinical reasoning and overcome the limitations of handling longitudinal clinical data ambiguity. METHODS: We retrospectively studied 3053 CRC patients (2010-2018). A local Large Language Model (LLM) (Qwen3:14B) analyzed ∼ 19,900 pathology and ∼ 43,900 imaging reports using iterative, patch-based analysis ("raw LLM"). Clinical validation rules were applied to generate "rule-based LLM" output, enhancing explainability and trustworthiness. Both automated methods and the manual Taiwan Cancer Registry (TCR) database were compared against a 20% manual reference standard (N = 602). Full prompts and validation code are provided for complete reproducibility. RESULTS: Under a strict 60-day temporal tolerance, the Rule-Based LLM achieved 90.7% accuracy, comparable to standard TCR processes (92.0%), and 77.2% sensitivity. The application of clinical validation rules significantly improved specificity from 87.7% (Raw LLM) to 93.9% (Rule-Based LLM). In time window analysis, the Rule-Based LLM identified 87.1% of recurrences within 60 days of the reference date. CONCLUSION: Our locally deployed, privacy-preserving, and explainable Clinical Context Engineering framework offers a viable, non-inferior alternative to standard TCR processes, reducing workload while maintaining data quality and fostering trust in AI-assisted cancer registry automation.

References

1. 1.Yang, Y. W., Chang, C. Y., Lin, Y. Z., Cheng, H. H., Huang, S. C., Lin, H. H., Lin, C. C., Lan, Y. T., Wang, H. S., Chang, S. C., Yang, S. H., Chen, W. S., & Jiang, J. K. (2026). Automated Flow and local LLM-Driven clinical Context Engineering: Precision colorectal cancer recurrence registry. International Journal of Medical Informatics, 106383. https://doi.org/10.1016/j.ijmedinf.2026.106383