Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 3 appraisals
Philosophical transactions. Series A, Mathematical, physical, and engineering sciences
A sentence is worth a thousand pictures: can large language models understand hum4n L4ngu4ge and the W0rld behind W0rds?
The current generation of large language models (LLMs) has been linked to claims about human-like linguistic performance, and their applications are hailed both as a step towards artificial general intelligence and as a major advance in understanding the cognitive and even neural basis of human language. To assess these claims, first, we analysed the contribution of LLMs as theoretically informative representations of a target cognitive system versus atheoretical mechanistic tools. Second, we evaluated the models' ability to see the bigger picture through top-down feedback from higher levels of processing, which requires grounding in previous expectations and past world experience. We hypothesize that since models lack grounded cognition, they cannot take advantage of these features and instead solely rely on fixed associations between represented words and word vectors. To assess this, we ran a novel leet task (l33t t4sk), which requires decoding sentences in which letters are systematically replaced by numbers. In line with our hypothesis, the results suggest that humans excel in this task, whereas models struggle. We interpret these results by identifying the key abilities that are still missing from the current state of development of these models, which require solutions that go beyond increased system scaling. This article is part of the theme issue 'World models in natural and artificial intelligence'.
15 May 2026
Read appraisal →Behavior research methods
A stroke-level large-scale database of Chinese character handwriting and the OpenHandWrite_Toolbox for handwriting research
Understanding what linguistic components (e.g., phonological, semantic, and orthographic systems) modulate Chinese handwriting at the character, radical, and stroke levels remains an important yet understudied topic. Additionally, there is a lack of comprehensive tools for capturing and batch-processing fine-grained handwriting data. To address these issues, we constructed a large-scale handwriting database in which 42 Chinese speakers each handwrote 1200 characters in a handwriting-to-dictation task. Additionally, we enhanced the existing handwriting package and provided comprehensive documentation for the upgraded OpenHandWrite_Toolbox. This toolbox allows users to easily modify the experimental design, capture the stroke-level handwriting trajectory, and batch-process handwriting measurements (e.g., latency and duration). Analysis of our large-scale database indicates that orthographic predictors affect handwriting preparation and execution at the character, radical, and stroke levels. Phonological factors influence handwriting at character and radical levels. Importantly, these lexical effects demonstrate hierarchical attenuation: they were most pronounced at the character level, followed by the radical, and weakest at the stroke level. These findings demonstrate that handwriting preparation and execution at the radical and stroke levels are closely intertwined with linguistic components. This database and toolbox offer valuable resources for future psycholinguistic and neurolinguistic research on the handwriting of characters and sub-characters across different languages.
14 May 2026
Read appraisal →Proceedings of the National Academy of Sciences of the United States of America
Discovering regularity and mechanisms of word sense acquisition in childhood.
How does language use inform the emergence of word meanings in early life? Prior work in developmental psychology and the cognitive sciences typically focuses on studying word acquisition in children without specifying how different senses within a word emerge through time. To shed light on word sense acquisition, we propose a framework grounded in state-of-the-art computational methodologies of contextual word embedding to characterize how different senses of a word unfold as children acquire their lexicon. Our framework identifies word senses automatically by forming semantic clusters through natural language use, and analyzes 1,270 words from approximately 4 million utterances produced by children (19 to 144 mo) and their caretakers. The psychological validity of these senses was assessed based on a combination of dictionaries, human judgment of semantic similarity, and evaluation from a large language model. We then tested three hypotheses motivated by existing work on word sense emergence in language evolution. First, concrete senses of a word tend to emerge earlier than more abstract senses in child language. Second, word senses grow incrementally in semantic space across development. Third, algorithms of semantic chaining-how words spawn new senses by extending from existing senses-recapitulate the order of word senses in development. We find support for all three hypotheses. Our work suggests that the development of word meaning in language acquisition resembles that in language evolution and offers a converging view on the cognitive principles and mechanisms across timescales in the ontogeny and phylogeny of word sense emergence.
13 May 2026
Read appraisal →