Chatbot Usability Scale in Chinese Users: Cross-Cultural Adaptation and Validation Study

Clinical Snapshot

Key Findings

• P Value: Not specifically reported for factor analysis

• Effect Size: Content validity index: 0.92, explaining 56.1% of variance in three-factor model

• Primary Outcome: Successful validation of Chinese BUS-11 with strong psychometric properties

• Nnt Or Sensitivity: Internal consistency reliability: Cronbach's α = 0.92 (excellent)

• Confidence Interval: Not reported for primary psychometric measures

Clinical Application

Highly feasible - concise 11-item scale that is user-friendly and culturally appropriate Limited direct applicability to Australian healthcare, but methodology could inform adaptation for multicultural populations and Chinese-Australian communities using digital health tools Chinese-speaking users and researchers evaluating chatbot usability in healthcare and other domains

Abstract

BACKGROUND: Chatbots are increasingly deployed across various domains; however, systematic evaluation of their usability remains limited, particularly in non-Western contexts. The 11-item Chatbot Usability Scale (BUS-11), a multidimensional instrument grounded in human-computer interaction theory, has demonstrated strong psychometric properties in prior studies; however, a validated Chinese version does not exist, despite China being one of the largest chatbot markets. OBJECTIVE: This study aimed to translate, culturally adapt, and validate the BUS-11 for Chinese users. METHODS: Following established cross-cultural adaptation procedures, the scale was forward- and back-translated, reviewed by an expert committee, and pilot-tested for clarity and feasibility. A main validation study was then conducted with 214 participants, who completed 438 evaluations of the chatbots across 10 widely used systems. RESULTS: Psychometric analyses demonstrated excellent scale-level content validity index (ie, 0.92), strong internal consistency (Cronbach α=0.92), and a clear 3-factor structure (Accessibility, Interaction Process Quality, and Information Quality), explaining 56.1% of the variance. Meanwhile, privacy or security and response time were retained as single-item indicators. The Chinese BUS-11 was concise, user-friendly, and psychometrically robust. CONCLUSIONS: This work fills a critical gap by providing the first validated instrument for assessing chatbot usability in Chinese contexts, enabling reliable cross-cultural comparisons and supporting both research and practical design evaluation in human-computer interaction.

References

1. 1.Ma, H., Pei, R., Li, S., Wang, A., Tang, X., & Piao, M. (2026). Chatbot Usability Scale in Chinese Users: Cross-Cultural Adaptation and Validation Study. JMIR Human Factors. https://doi.org/10.2196/84971