TRUST: an large language model-based dialogue system for trauma understanding and structured assessments

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Not reported in abstract; qualitative comparability claim only

• Primary Outcome: Expert evaluation by conversation and clinical specialists indicating that TRUST performs comparably to real-life clinical PTSD diagnostic interviews conducted by human clinicians

• Nnt Or Sensitivity: Diagnostic accuracy metrics (sensitivity, specificity, PPV, NPV) not reported; no NNT calculable at this stage of development

• Confidence Interval: Not reported

Clinical Application

Technically feasible as a supplementary or triage tool in well-resourced digital health environments. Significant barriers to clinical deployment remain: regulatory approval, integration with electronic medical records, clinician oversight requirements, data privacy compliance, and liability frameworks. Requires substantial prospective clinical validation before any patient-facing deployment. Australia faces a well-documented shortage of mental health clinicians, particularly in rural, remote, and Indigenous communities, making the concept of LLM-assisted PTSD assessment potentially relevant. However, several Australian-specific considerations apply: (1) The TGA would likely classify such a system as a Software as a Medical Device (SaMD) requiring regulatory assessment under the TGA's Digital Health framework before clinical use. (2) The RACGP and RANZCP have not yet issued guidance on LLM-assisted diagnostic interviewing. (3) The Australian veteran population (DVA clients) represents a high-priority PTSD cohort, but the Phoenix Australia PTSD guidelines emphasise clinician-led assessment. (4) Cultural safety for Aboriginal and Torres Strait Islander peoples — for whom trauma-informed, culturally responsive care is essential — cannot be assumed from a US-developed LLM system without specific validation. (5) Privacy obligations under the Australian Privacy Act 1988 and My Health Record Act 2012 would apply to any deployment involving patient data. Potentially applicable to adults requiring PTSD screening or structured diagnostic assessment in settings with limited access to trained clinicians — including telehealth, rural and remote services, veteran health services, and high-volume trauma clinics. Not yet validated for paediatric populations, non-English speakers, or individuals with comorbid psychosis or intellectual disability.

Abstract

OBJECTIVES: While large language models (LLMs) have been widely used to assist clinicians and support patients, no existing work has explored dialogue systems for standard diagnostic interviews and assessments. This study aims to bridge the gap in mental healthcare accessibility by developing an LLM-powered dialogue system that replicates clinician behavior. MATERIALS AND METHODS: We introduce TRUST, a framework of cooperative LLM modules capable of conducting formal diagnostic interviews and assessments for post-traumatic stress disorder (PTSD) following the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5). To guide the generation of appropriate clinical responses, we propose a Dialogue Acts schema specifically designed for clinical interviews. Additionally, we develop a patient simulation approach based on real-life interview transcripts to replace time-consuming and costly manual testing by clinicians. RESULTS: A comprehensive set of evaluation metrics is designed to assess the dialogue system from both the agent and patient simulation perspectives. Expert evaluations by conversation and clinical specialists show that TRUST performs comparably to real-life clinical interviews. DISCUSSION: Our system performs with clinical quality approaching that of human clinicians, with room for future enhancements in communication styles and response appropriateness. CONCLUSIONS: Our TRUST framework shows its potential to facilitate mental healthcare availability.

References

1. 1.Tu, S., Powers, A., Doogan, S., & Choi, J. D. (2023). TRUST: An LLM-based dialogue system for trauma understanding and structured assessments. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 15750–15762). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.acl-long.870
2. 2.Weathers, F. W., Bovin, M. J., Lee, D. J., Sloan, D. M., Schnurr, P. P., Kaloupek, D. G., Keane, T. M., & Marx, B. P. (2018). The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5): Development and initial psychometric evaluation in military veterans. Psychological Assessment, 30(3), 383–395. https://doi.org/10.1037/pas0000486