Large language models accurately identify decision reasons in verbal reports.

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract

• Effect Size: 95% trial-level alignment between LLM-identified reasons and actual choices; reason-based representations described as more parsimonious and informative than choice-only inferences; problem structure accounts for more variance in decision reasons than individual differences

• Primary Outcome: LLM-identified decision reasons from free-text verbal reports aligned with participants' actual lottery choices in 95% of trials

• Nnt Or Sensitivity: Sensitivity and specificity for individual reason categories not reported in abstract; out-of-sample prediction accuracy described as 'competitive with established computational models' (specific figures not available from abstract)

• Confidence Interval: Not reported in abstract

Clinical Application

The LLM-based framework is described as scalable, suggesting feasibility for large-scale research applications. However, clinical implementation would require: (1) revalidation in healthcare decision contexts; (2) assessment of performance across languages and health literacy levels; (3) ethical and governance frameworks for processing patient verbal data through LLM systems; (4) integration with existing clinical research infrastructure. Current feasibility for direct clinical practice is low. This study has no direct immediate application to Australian clinical practice, PBS prescribing, or TGA-regulated interventions. However, it has potential relevance to: (1) RACGP-aligned shared decision-making research, where understanding patient decision reasoning could improve consultation quality; (2) Australian health technology assessment (PBAC/MSAC) processes that incorporate patient preference data; (3) health behaviour research funded by NHMRC examining decision-making in preventive health contexts. Any application in Australian healthcare would require revalidation with Australian patient populations, consideration of Indigenous health decision-making contexts, and compliance with Australian Privacy Act requirements for processing verbal/text data through offshore LLM systems. Researchers and methodologists in decision science, behavioural economics, and cognitive psychology studying human choice under risk; potential future application to clinical shared decision-making research if framework is validated in healthcare contexts

Abstract

Understanding the reasons behind human choices under risk is a central goal of decision scientists, but traditional methods relying on behavioral data are limited by strict invariance assumptions. We introduce a scalable analytical framework using large language models (LLMs) to analyze verbal reports and identify articulated reasons for choice between monetary lotteries. A validated LLM accurately identified predefined decision reasons in participants' free-text reports, aligning with their actual choices in 95% of trials. Our analysis reveals that the reasons behind people's decisions vary systematically and are driven more by the structure of the choice problem than by individual differences. Crucially, reasons identified from verbal reports yield more parsimonious and informative representations of decision processes compared to those inferred from choices alone; furthermore, problem-specific reason profiles achieve out-of-sample prediction accuracy that is competitive with established computational models. This work demonstrates that verbal reports are a rich data source and our analytical framework can unlock their potential, delivering results that challenge the field's foundational invariance assumptions and pave the way for more context-sensitive and interpretable models of human decision making.

References

1. 1.Fuławka, K., Hertwig, R., & Wulff, D. U. (2026). Large language models accurately identify decision reasons in verbal reports. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2526798123