Large Language Model Versus Multidisciplinary Team: Feasibility Study of Pancreatic Cancer Management Recommendations
Clinical Snapshot
PICO Framework
| P — Population | Adults with pancreatic cancer reviewed by a multidisciplinary team (MDT) at People's Hospital of Leshan and West China Hospital of Sichuan University, China (n=125 complete structured cases from 269 MDT cases) |
| I — Intervention | Large language model (chat interface labelled ChatGPT-5.2) generating broad initial management category recommendations from structured case inputs |
| C — Comparator | Multidisciplinary team (MDT) consensus management recommendations serving as the reference standard |
| O — Outcomes | Primary: concordance between LLM-generated and MDT-recommended broad initial management category (binary match/no-match) |
Bottom Line
This feasibility study reports 95.2% concordance between an LLM and MDT recommendations across 125 pancreatic cancer cases, a superficially impressive figure that is substantially undermined by critical methodological limitations. The LLM is identified only as 'ChatGPT-5.2' — a designation with no publicly verifiable documentation — and was explicitly not version controlled, rendering the study irreproducible. Over half of eligible cases were excluded without explanation, raising serious selection bias concerns. The primary outcome ('broad initial management category') is undefined and coarse, almost certainly inflating apparent concordance. No blinding, inter-rater reliability, or subgroup analyses are reported. The authors appropriately limit their conclusions to 'supervised exploratory use', which is the correct clinical posture. For Australian HPB and oncology clinicians, this study provides no actionable evidence to support LLM integration into pancreatic cancer MDT workflows. MDT decision-making in this disease requires granular, patient-specific judgement encompassing staging, comorbidity, patient preference, and PBS/clinical trial access — dimensions entirely absent from this evaluation. Senior clinicians should treat this as hypothesis-generating only.
Key Findings
P Value: Not reported
Effect Size: 119/125 cases concordant = 95.2% agreement rate
Primary Outcome: Concordance between LLM ('ChatGPT-5.2') and MDT broad initial management category recommendation across 125 structured pancreatic cancer cases
Nnt Or Sensitivity: No sensitivity, specificity, kappa, or NNT reported. Concordance proportion only. Chance-level baseline not established, limiting interpretability of the 95.2% figure.
Confidence Interval: 95% CI 89.8% to 98.2%
Clinical Application
Clinical implementation is not feasible based on this evidence alone. The LLM version is unverifiable and non-reproducible; the outcome measure does not reflect the granularity of real MDT decision-making (e.g., specific chemotherapy regimen, surgical approach, palliative intent, clinical trial eligibility). No safety data, error characterisation, or failure mode analysis is presented. Supervised exploratory use in research settings only, as the authors themselves recommend. In Australia, pancreatic cancer MDT meetings are mandated under Cancer Australia and state-based cancer network guidelines (e.g., NSW Cancer Institute, Victorian Comprehensive Cancer Centre). RACGP and COSA guidelines emphasise multidisciplinary care as the standard of care. PBS-listed treatments for pancreatic cancer (e.g., gemcitabine, FOLFIRINOX components, nab-paclitaxel via Section 100) require nuanced, patient-specific decision-making that a 'broad category' LLM output cannot replicate. TGA has not approved any LLM-based clinical decision support tool for oncology. Australian clinicians should not alter MDT practice based on this feasibility study. The study's findings are not generalisable to Australian case mix, staging workflows, or treatment access frameworks. Patients with pancreatic cancer undergoing MDT review in tertiary hepato-pancreato-biliary surgical centres. Findings are not directly applicable to community oncology, regional hospitals, or settings outside the study's Chinese tertiary context.
Abstract
Of 269 pancreatic cancer multidisciplinary team cases, 125 complete structured cases were evaluated. The chat interface model labeled ChatGPT-5.2 matched the multidisciplinary team's broad initial management category in 119 cases (95.2%, 95% CI 89.8%-98.2%). The end point was coarse, and the model was not version controlled, so findings only support supervised exploratory use.
References
- 1.Liu, Z., Zhao, X., Mao, T., Zhou, H., & Jiang, K. (2026). Large language model versus multidisciplinary team: Feasibility study of pancreatic cancer management recommendations. Journal of Medical Internet Research. https://doi.org/10.2196/95411
Related Research
The American journal of pathology
An Integrated Machine Learning and Genomic Framework for Precise Detection of Gastric Cancer
3 Aug 2026
Current opinion in chemical biology
Prediction of protein-protein interactions and co-complex models with deep learning
3 Aug 2026
European journal of radiology
A CT-based deep learning model to differentiate between benign and malignant adrenal lesions
3 Aug 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service