Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 8 appraisals
Journal of medical Internet research
Using Ultra-Abridged Individual Difference Scales for Personalization in Digital Mental Health to Improve Uptake, Engagement, and Experiences: Three-Tiered Decision Framework for Scale Shortening
Given the diversity of human characteristics and experiences, personalization in nudges, messages, choice presentations, interventions, and overall product design has been increasingly adopted in digital health to promote engagement. Past studies on moderators and personalization in digital health and mental health services generally focused on demographic and symptom variables, with generally inconsistent findings or null findings. Cognitive, motivational, and decisional psychological attributes are largely overlooked. Psychology often uses long self-report scales to measure various psychological attributes. Although they are useful in tapping into individuals' psychological profiles, when applied in real-life, everyday settings to assess individual differences, people are most likely unwilling to complete them. With the pressing need to personalize digital health platforms to enhance uptake, retention, and engagement, ultrashort versions of these psychological scales may be considered to allow assessment of multiple attributes at the same time. Scale shortening can be achieved through regression analyses of each item, factor analyses, item response theory, ant colony optimization, and machine learning methods, with each method having advantages, disadvantages, and conditions required to make it suitable. To illustrate, we provided examples of regression analyses of each item and factor analyses, with potential implications for personalizing narrative versus research-based messages in digital mental health contexts. We present a 3-tiered decision framework for scale shortening method selection depending on goals and possible constraints, with guidelines on validation methods for ultrashort scales. Moving forward, more validation studies and field studies in digital health platforms are needed to evaluate the ecological validity, reliability, and generalizability of these methods, bearing in mind the limitations and conditions where such shortening methods may not work well. Researchers may compare the effectiveness and limitations of personalization using ultrashort scales with other commonly adopted personalization methods (eg, based on longer scales, behavioral data, and large language models). Ethical concerns need to be considered and mitigated carefully, respecting diverse preferences, informed choices, and the privacy of service users. Our viewpoint piece is primarily intended for digital mental health researchers and practitioners, but may also be informative for the fields of digital health and medicine as well as personalization (eg, personalized health care, personalized nudging, and message matching) more broadly, given the common goal of boosting uptake and engagement as well as improving service users' experiences.
10 June 2026
Read appraisal →Behavior research methods
Neural cognitive diagnosis modeling incorporating response times
Cognitive diagnosis is a fundamental issue in the field of intelligent education, aiming to identify students' mastery of specific knowledge concepts. With computerized testing, response time (RT) is a process data that can be collected. Incorporating RT in cognitive diagnosis assessment can enhance diagnostic accuracy. However, RT is only considered in traditional statistical cognitive diagnosis models. Compared with traditional statistical diagnostic models, cognitive diagnosis models based on neural networks have advantages such as high precision and strong generalization ability. Therefore, this paper proposes a JRT-NCD (joint response times neural cognitive diagnosis) model that uses neural networks to model the complex nonlinear interactions between exercises and students and incorporates RT as a new feature to refine diagnostic results on student abilities. Research findings on three datasets of PISA2012, 2MFC, and ASSIST09 indicate that: (1) In comparison with traditional statistical models, neural networks have better fitting capabilities for real complex nonlinear data; (2) compared to the NCD model that disregards RT, JRT-NCD achieves higher diagnostic accuracy while maintaining its interpretability, and reduces the misleading effects of "overspeed behavior" on the diagnostic results.
27 Apr 2026
Read appraisal →PloS one
Research on the development of an automated system for psychology questionnaire generation based on large language models
This study reimagined the psychology questionnaire development process using large language model ((LLM) technology, aiming to overcome the protracted preparation cycles and significant human bias inherent in traditional scale development. We developed a specialized fine-tuning scheme for a corpus of 169 professional psychological questionnaires. By integrating instruction fine-tuning with human feedback reinforcement, we significantly enhanced the adaptability of the Qwen-2.5 and GLM-4 models for demanding professional psychological assessment tasks. The optimized models demonstrated remarkable gains across key dimensions: text generation quality (BLEU-4 increased by 0.05, ROUGE-L by 0.057), scientific rigor (logical consistency improved by 28.6%), and cultural adaptability (achieving over 85% accuracy in cross-regional expression conversion). This research solidly supports the feasibility of leveraging LLM technology to drive research paradigm transformation in psychology, offering crucial methodological support for developing efficient, intelligent psychological measurement tools.
27 Apr 2026
Read appraisal →JMIR human factors
Psychometric Properties of Digital Divide Measurement Instruments in Health Contexts: Systematic Review
BACKGROUND: The digital divide constitutes a growing form of social inequality, influencing access to, use of, and benefit from digital health technologies. Accurate assessment of this phenomenon requires psychometrically sound measurement instruments, especially in health care contexts. OBJECTIVE: This systematic review aimed to identify and evaluate the psychometric properties of instruments developed to measure the digital divide, with a particular focus on their application in health-related settings. METHODS: A systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 and COSMIN (Consensus-Based Standards for the Selection of Health Measurement Instruments) guidelines. Literature searches were performed in PubMed, Scopus, Dialnet, and Web of Science for studies published between 2015 and 2025. Inclusion criteria required studies to report a psychometric evaluation of instruments measuring the digital divide. RESULTS: Four studies met the inclusion criteria (ranging from 875 to 1337 participants). The identified scales primarily measured digital skills and capital. All studies reported adequate internal consistency, with Cronbach α coefficients ranging from 0.70 to 0.94. Structural validity was confirmed through factor analysis in all studies. However, data on temporal stability and responsiveness were absent. CONCLUSIONS: Existing instruments show potential for assessing digital inequality in health, but further psychometric validation and cultural adaptation are required. Nursing professionals are encouraged to lead efforts in developing and validating context-sensitive instruments to promote digital health equity.
25 Apr 2026
Read appraisal →Sleep & breathing = Schlaf & Atmung
Health-related quality of life in patients with obstructive sleep apnea - a systematic review update assessing the quality of measurement properties of patient-reported outcome measures
PURPOSE: The purpose of this systematic review update was to identify existing health-related quality of life (HRQOL) patient-reported outcome measures (PROMs) for obstructive sleep apnea (OSA). Building upon a 2016 systematic review, which recommended four PROMs with the highest evidence, this investigation was warranted to synthesize new evidence and evaluate new PROMs that have emerged since then via the Consensus based Standards for the Selection of Health Measurement Instruments (COSMIN). METHODS: We searched Medline via PubMed, Embase and the Cochrane Library for HRQOL PROM development and validation studies published in English or German. Studies focusing on cross-cultural validation, those in which PROMs were assessed as secondary outcomes, and studies limited to specific demographic subgroups or single HRQOL domains were excluded. For risk of bias and quality of measurement properties (MP) assessment, we used the COSMIN methodology to facilitate comparability with the previous review, as well as the GRADE approach for evidence level evaluation. RESULTS: We identified two new PROMs, the Patient-Reported Apnea Questionnaire (PRAQ) and the Symptoms, Tiredness, Alertness, Mood and Psychosocial instrument (STAMP), which were validated in two studies and a single study, respectively. Overall, the PRAQ displays an moderate risk of bias with excellent quality of MP and high evidence for content validity in particular, whereas the STAMP shows a higher risk of bias and low quality of MP. CONCLUSION: The PRAQ has demonstrated the strongest evidence among PROMs since 2016. In total, including the previous review, five PROMs are currently recommended for assessing HRQOL in individuals with OSA. REVIEW REGISTRATION: The review was registered with PROSPERO under the registration number CRD42024602020.
24 Apr 2026
Read appraisal →Behavior research methods
A modified expectation-maximization algorithm for accelerated item response theory model estimation with large datasets
The expectation-maximization (EM) algorithm is widely used for parameter estimation in item response theory (IRT) modeling. However, when applied to datasets with large numbers of individuals and items, the standard EM algorithm can be slow to converge, with computationally expensive E-steps. We propose a modified EM algorithm to accelerate estimation for unidimensional two-parameter logistic IRT models. The modified algorithm uses a two-stage structure with partial-step updating over data subsets to reduce convergence time, while maintaining comparable accuracy and precision. The first two simulation studies evaluated its performance relative to standard EM, focusing on convergence time, parameter recovery, and standard error estimation across varying subset sizes and item counts. The third study demonstrated its scalability and runtime advantage in a large-scale testing scenario involving one million respondents and 100 items. The fourth study evaluated robustness under departures from unidimensionality. The proposed algorithm showed time reductions under smaller subsets with 36 items, consistent reductions across all subset sizes with 54 items, and the largest reduction (60%) with 40-item forms constructed from 100 items, while maintaining comparable estimation performance. These results highlight the algorithm's potential for large-scale applications involving tens of thousands of respondents and moderate-to-large item pools, with modifications that can be integrated into existing EM routines. We conclude with directions for operational use and potential extensions to multidimensional IRT estimation.
23 Apr 2026
Read appraisal →Psychological medicine
What is being measured by formal thought disorder scales? An item-level content analysis.
BACKGROUND: Formal thought disorder (FTD), characterized by disruptions in the flow and form of thought, is a core feature of psychosis. But its measurement is fragmented across numerous rating scales, leading to its continued neglect in both research and clinical practice. To determine if different FTD scales measure the same underlying construct, we need to assess the degree to which the content of commonly used FTD scales overlaps with each other. METHODS: We conducted a systematic review to identify all standardized, clinician-rated scales used to measure FTD in psychotic disorders. From this set, we extracted individual items and derived a consensus list of 56 discrete FTD phenomena. Two independent clinical experts conducted item-to-item mapping for every scale item onto these FTD phenomena. Content overlap between scales was quantified using Jaccard Similarity Index (JSI). We determined the overall coverage achieved via several combinations of FTD scales. RESULTS: The 15 scales, comprising 207 items, showed weak content overlap. The mean JSI across all scale pairs was low, and no single phenomenon featured across all scales. While some core FTD phenomena (e.g. 'incoherence', 'poor speech content', 'drifting-off') were represented in many scales, 20% of all identified features were idiosyncratic, appearing in only one scale. CONCLUSIONS: Existing FTD rating scales capture a wide but heterogeneous array of symptoms with poor content overlap. This lack of harmonization challenges the comparability of mechanistic and interventional studies. We highlight the need for a consensus-based, standardized measurement of FTD and provide a comprehensive checklist to advance the research and clinical practice.
14 Apr 2026
Read appraisal →JMIR human factors
Chatbot Usability Scale in Chinese Users: Cross-Cultural Adaptation and Validation Study
BACKGROUND: Chatbots are increasingly deployed across various domains; however, systematic evaluation of their usability remains limited, particularly in non-Western contexts. The 11-item Chatbot Usability Scale (BUS-11), a multidimensional instrument grounded in human-computer interaction theory, has demonstrated strong psychometric properties in prior studies; however, a validated Chinese version does not exist, despite China being one of the largest chatbot markets. OBJECTIVE: This study aimed to translate, culturally adapt, and validate the BUS-11 for Chinese users. METHODS: Following established cross-cultural adaptation procedures, the scale was forward- and back-translated, reviewed by an expert committee, and pilot-tested for clarity and feasibility. A main validation study was then conducted with 214 participants, who completed 438 evaluations of the chatbots across 10 widely used systems. RESULTS: Psychometric analyses demonstrated excellent scale-level content validity index (ie, 0.92), strong internal consistency (Cronbach α=0.92), and a clear 3-factor structure (Accessibility, Interaction Process Quality, and Information Quality), explaining 56.1% of the variance. Meanwhile, privacy or security and response time were retained as single-item indicators. The Chinese BUS-11 was concise, user-friendly, and psychometrically robust. CONCLUSIONS: This work fills a critical gap by providing the first validated instrument for assessing chatbot usability in Chinese contexts, enabling reliable cross-cultural comparisons and supporting both research and practical design evaluation in human-computer interaction.
7 Apr 2026
Read appraisal →