A sentence is worth a thousand pictures: can large language models understand hum4n L4ngu4ge and the W0rld behind W0rds?

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Not reported

• Primary Outcome: Comparative performance on leet decoding task between humans and LLMs

• Nnt Or Sensitivity: Qualitative finding: humans excel while LLMs struggle with systematic letter-to-number substitution decoding

• Confidence Interval: Not reported

Clinical Application

Not applicable to clinical practice No direct relevance to Australian healthcare, though findings may inform AI implementation in clinical decision support systems Not applicable - theoretical cognitive science research

Abstract

The current generation of large language models (LLMs) has been linked to claims about human-like linguistic performance, and their applications are hailed both as a step towards artificial general intelligence and as a major advance in understanding the cognitive and even neural basis of human language. To assess these claims, first, we analysed the contribution of LLMs as theoretically informative representations of a target cognitive system versus atheoretical mechanistic tools. Second, we evaluated the models' ability to see the bigger picture through top-down feedback from higher levels of processing, which requires grounding in previous expectations and past world experience. We hypothesize that since models lack grounded cognition, they cannot take advantage of these features and instead solely rely on fixed associations between represented words and word vectors. To assess this, we ran a novel leet task (l33t t4sk), which requires decoding sentences in which letters are systematically replaced by numbers. In line with our hypothesis, the results suggest that humans excel in this task, whereas models struggle. We interpret these results by identifying the key abilities that are still missing from the current state of development of these models, which require solutions that go beyond increased system scaling. This article is part of the theme issue 'World models in natural and artificial intelligence'.

References

1. 1.Leivada, E., Marcus, G., Günther, F., & Murphy, E. (2026). A sentence is worth a thousand pictures: can large language models understand hum4n L4ngu4ge and the W0rld behind W0rds? Philosophical Transactions of the Royal Society A, 384(2274). https://doi.org/10.1098/rsta.2025.0008