Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 2 appraisals
PloS one
Research on the development of an automated system for psychology questionnaire generation based on large language models
This study reimagined the psychology questionnaire development process using large language model ((LLM) technology, aiming to overcome the protracted preparation cycles and significant human bias inherent in traditional scale development. We developed a specialized fine-tuning scheme for a corpus of 169 professional psychological questionnaires. By integrating instruction fine-tuning with human feedback reinforcement, we significantly enhanced the adaptability of the Qwen-2.5 and GLM-4 models for demanding professional psychological assessment tasks. The optimized models demonstrated remarkable gains across key dimensions: text generation quality (BLEU-4 increased by 0.05, ROUGE-L by 0.057), scientific rigor (logical consistency improved by 28.6%), and cultural adaptability (achieving over 85% accuracy in cross-regional expression conversion). This research solidly supports the feasibility of leveraging LLM technology to drive research paradigm transformation in psychology, offering crucial methodological support for developing efficient, intelligent psychological measurement tools.
27 Apr 2026
Read appraisal →Psychological medicine
What is being measured by formal thought disorder scales? An item-level content analysis.
BACKGROUND: Formal thought disorder (FTD), characterized by disruptions in the flow and form of thought, is a core feature of psychosis. But its measurement is fragmented across numerous rating scales, leading to its continued neglect in both research and clinical practice. To determine if different FTD scales measure the same underlying construct, we need to assess the degree to which the content of commonly used FTD scales overlaps with each other. METHODS: We conducted a systematic review to identify all standardized, clinician-rated scales used to measure FTD in psychotic disorders. From this set, we extracted individual items and derived a consensus list of 56 discrete FTD phenomena. Two independent clinical experts conducted item-to-item mapping for every scale item onto these FTD phenomena. Content overlap between scales was quantified using Jaccard Similarity Index (JSI). We determined the overall coverage achieved via several combinations of FTD scales. RESULTS: The 15 scales, comprising 207 items, showed weak content overlap. The mean JSI across all scale pairs was low, and no single phenomenon featured across all scales. While some core FTD phenomena (e.g. 'incoherence', 'poor speech content', 'drifting-off') were represented in many scales, 20% of all identified features were idiosyncratic, appearing in only one scale. CONCLUSIONS: Existing FTD rating scales capture a wide but heterogeneous array of symptoms with poor content overlap. This lack of harmonization challenges the comparability of mechanistic and interventional studies. We highlight the need for a consensus-based, standardized measurement of FTD and provide a comprehensive checklist to advance the research and clinical practice.
14 Apr 2026
Read appraisal →