Research on the development of an automated system for psychology questionnaire generation based on large language models
Clinical Snapshot
PICO Framework
| P — Population | Psychology questionnaire development process and existing psychological assessment instruments |
| I — Intervention | Automated system using fine-tuned large language models (Qwen-2.5 and GLM-4) with instruction fine-tuning and human feedback reinforcement |
| C — Comparator | Traditional manual psychology questionnaire development methods |
| O — Outcomes | Text generation quality (BLEU-4, ROUGE-L scores), scientific rigor (logical consistency), cultural adaptability, and feasibility of LLM-driven questionnaire generation |
Bottom Line
This technology development study presents an automated system for generating psychology questionnaires using large language models, showing improvements in text generation metrics and logical consistency. However, the research lacks crucial validation elements for clinical application. While the technical approach appears sound, there is insufficient evidence of psychological construct validity, clinical utility, or comparison with expert-developed instruments. The study represents an interesting proof-of-concept but requires substantial additional validation before clinical implementation. Australian psychologists should await peer-reviewed validation studies demonstrating that AI-generated questionnaires maintain the psychometric properties essential for reliable psychological assessment. The technology may have future potential for supporting questionnaire development workflows, but cannot replace expert psychological assessment design without proper validation.
Key Findings
P Value: Not reported
Effect Size: BLEU-4 increased by 0.05, ROUGE-L by 0.057, logical consistency improved by 28.6%
Primary Outcome: Automated psychology questionnaire generation system performance
Nnt Or Sensitivity: Over 85% accuracy in cross-regional expression conversion
Confidence Interval: Not reported
Clinical Application
Limited without proper validation against established psychological constructs and clinical outcomes Would require validation against Australian psychological practice standards, cultural considerations, and integration with existing assessment frameworks used by psychologists Psychology researchers and clinicians developing assessment instruments
Abstract
This study reimagined the psychology questionnaire development process using large language model ((LLM) technology, aiming to overcome the protracted preparation cycles and significant human bias inherent in traditional scale development. We developed a specialized fine-tuning scheme for a corpus of 169 professional psychological questionnaires. By integrating instruction fine-tuning with human feedback reinforcement, we significantly enhanced the adaptability of the Qwen-2.5 and GLM-4 models for demanding professional psychological assessment tasks. The optimized models demonstrated remarkable gains across key dimensions: text generation quality (BLEU-4 increased by 0.05, ROUGE-L by 0.057), scientific rigor (logical consistency improved by 28.6%), and cultural adaptability (achieving over 85% accuracy in cross-regional expression conversion). This research solidly supports the feasibility of leveraging LLM technology to drive research paradigm transformation in psychology, offering crucial methodological support for developing efficient, intelligent psychological measurement tools.
References
- 1.Yuan, Z., Jia, C., Lan, M., Zhao, L., Chen, Z., Yang, M., Liu, X., Ni, N., & Wu, S. (2024). Research on the development of an automated system for psychology questionnaire generation based on large language models. PLoS One. https://doi.org/10.1371/journal.pone.0345117
Related Research
International journal of yoga therapy
Randomized Controlled Trial on the Effect of Yoga on Anxiety Severity, Quality of Life, and Biomarkers in Individuals with Anxiety Disorder
31 July 2026
European journal of psychotraumatology
Differential linguistic patterns across Rorschach, TAT, and SCT in individuals with PTSD
27 July 2026
Journal of behavioral addictions
Efficacy of a mobile-based approach-avoidance task training (PROTECTapp) for problematic usage of the internet in young adults: A randomized controlled trial
18 July 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service