Reference-Free large language model agents for physician-guided radiotherapy treatment planning

Clinical Snapshot

Key Findings

• P Value: Dmax: p<0.05; Boost PTV conformity index: p<0.05; Primary PTV conformity index: p=0.47 (non-significant)

• Effect Size: Dmax: 106.5% (LLM) vs. 108.8% (clinical); Boost PTV conformity index: 1.18 (LLM) vs. 1.39 (clinical); Primary PTV conformity index: 1.82 (LLM) vs. 1.88 (clinical)

• Primary Outcome: LLM-generated IMRT plans achieved comparable OAR sparing to clinically approved plans, with statistically significant improvements in hot spot control (Dmax) and boost PTV conformity index

• Nnt Or Sensitivity: Not applicable for this dosimetric feasibility study; no NNT, sensitivity, specificity, or hazard ratio reported. Planning failure rate and LLM iteration efficiency metrics not disclosed.

• Confidence Interval: Not reported for any outcome

Clinical Application

Technical feasibility is demonstrated at proof-of-concept level only. Significant barriers to clinical implementation include: undisclosed LLM identity and cost, unknown planning time per case, absence of failure mode analysis, lack of regulatory approval pathways, and the requirement for seamless TPS API integration. Workflow integration into existing dosimetry pipelines would require substantial institutional investment and validation. In Australia, IMRT for HN cancer is delivered across public and private radiation oncology centres under RANZCR and ACPSEM guidelines. The TGA does not currently have a specific regulatory framework for LLM-based clinical decision support in radiotherapy planning; any clinical deployment would require TGA Software as a Medical Device (SaMD) assessment under the TGA's Digital Health framework. Medicare Benefits Schedule (MBS) items for radiation treatment planning (e.g., MBS 15275, 15278) are tied to qualified medical physicist and radiation oncologist oversight — LLM-generated plans would still require certified human review and approval. The ACPSEM and RANZCR would need to develop position statements on LLM integration before clinical adoption. Australian radiation oncology centres using Varian Eclipse, Elekta Monaco, or RayStation TPS would need vendor-specific API validation. This technology is not PBS-relevant at present but has future implications for workforce efficiency in centres experiencing dosimetrist shortages. Adult patients with HN cancer undergoing IMRT at institutions with compatible commercial TPS infrastructure and access to LLM API services. At this stage, applicability is restricted to research settings with robust physicist oversight. The technology is not ready for unsupervised clinical deployment.

Abstract

BACKGROUND: Large language models (LLMs) have recently demonstrated exceptional capabilities, offering the potential to streamline workflows and enhance efficiency across diverse tasks. However, their application in domain-specific areas, such as radiotherapy treatment planning, remains challenging due to the lack of publicly available, specialized knowledge required for these tasks. PURPOSE: To investigate the critical architectural and functional components necessary to adapt an off-the-shelf large language model into a clinically viable planning agent for inverse treatment planning in intensity-modulated radiation therapy (IMRT). MATERIALS/METHODS: Twenty head-and-neck (HN) cancer patients who received IMRT at our institution were retrospectively collected under IRB approval. The LLM agent was implemented to directly interact with the clinical treatment planning system (TPS) to iteratively extract intermediate plan states and propose new constraint values to guide inverse optimization. Its decision-making was informed by real-time plan evaluations and prior optimization outcomes, enabling adaptive refinement of planning strategies across iterations. The agent was equipped with three core capabilities: comprehension of clinical planning objectives, contextual understanding of the optimization environment, and arithmetic proficiency for quantitative reasoning. Treatment planning was conducted in a reference-free inference setting, wherein the LLM operated without prior exposure to manually generated treatment plans and without any fine-tuning or task-specific training. LLM-generated treatment plans were compared with clinically approved plans created by certified dosimetrists. Key dosimetric endpoints were evaluated and statistically analyzed. Paired comparisons between LLM-generated and clinical plans were performed using the Wilcoxon signed-rank test. RESULTS: LLM-generated plans achieved comparable organ-at-risk (OAR) sparing relative to clinical plans, while demonstrating improved hot spot control (Dmax: 106.5% vs. 108.8%, p < 0.05) and superior conformity (conformity index: 1.18 vs. 1.39, p < 0.05 for boost PTV; 1.82 vs. 1.88, p = 0.47 for primary PTV). CONCLUSIONS: This study demonstrates the feasibility of a reference-free, LLM-driven workflow for automated IMRT treatment planning in a commercial TPS. The proposed approach provides a generalizable solution that could reduce planning variability and support broader adoption of AI-based planning strategies.

References

1. 1.Yang, D., Wu, X., Xie, Y., Li, X., Wu, Q., Wu, Q. J., & Sheng, Y. (2026). Reference-Free large language model agents for physician-guided radiotherapy treatment planning. Medical Physics. https://doi.org/10.1002/mp.70592