Japanese Health Information Technology Usability Evaluation Scale for Sexually Transmitted Infection-Related Chatbots: Development and Psychometric Validation Study

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract; statistical significance of correlations implied but not explicitly stated

• Effect Size: Cronbach's alpha: 0.85–0.98 across subscales; convergent validity correlations: r=0.779 (satisfaction) and r=0.797 (usage intention)

• Primary Outcome: Psychometric validation of the Japanese Health-ITUES for an STI-related chatbot: internal consistency, construct validity (CFA), and convergent validity

• Nnt Or Sensitivity: Not applicable (psychometric validation study); CFA fit indices: RMSEA=0.10 (borderline acceptable), CFI>0.90 (acceptable)

• Confidence Interval: Not reported for any statistic

Clinical Application

The Japanese Health-ITUES is a low-burden, self-administered questionnaire suitable for integration into mHealth usability testing protocols, formative evaluation phases, and post-deployment user experience assessments. It requires no specialised equipment and can be administered digitally. However, test-retest reliability data are needed before longitudinal or repeated-measures use. Australia has a growing mHealth sector with increasing TGA interest in software as a medical device (SaMD) regulation. The RACGP supports digital health integration in primary care, and STI rates — particularly chlamydia — remain elevated in young Australians (AIHW data). A validated Japanese-language usability scale has limited direct PBS or TGA regulatory relevance but is pertinent to: (1) culturally and linguistically diverse (CALD) health service design for Japanese-Australian communities; (2) informing Australian researchers developing or evaluating STI-related mHealth tools who may adapt the Health-ITUES methodology; and (3) supporting the broader evidence base for usability evaluation frameworks applicable to TGA SaMD assessment processes. Direct clinical application in mainstream Australian practice is limited pending cross-cultural validation in Australian populations. Young Japanese women seeking STI-related health information via digital or mobile health platforms; potentially extensible to other Japanese-speaking populations using mHealth tools after further validation

Abstract

BACKGROUND: The rapid expansion of mobile technology has accelerated the integration of health applications and conversational AI into clinical and public health practices. To ensure these tools are effective and sustainable, usability evaluations and early user engagement during development are essential. The Health Information Technology Usability Evaluation Scale (Health-ITUES) is a validated and flexible usability assessment instrument that is available in multiple languages and applicable across diverse contexts. However, a Japanese version of this scale has not yet been developed. OBJECTIVE: This study aimed to translate and validate a Japanese version of the Health-ITUES, customized for a sexually transmitted infection (STI)-related chatbot, and to support the usability assessment of emerging mobile health tools in Japan. METHODS: We developed a Japanese version of the Health-ITUES using a chatbot under development as a consultation tool for young women regarding STIs. First, the original scale was customized to reflect the chatbot's specific purpose and intended usage context. Following established translation guidelines, we conducted forward translation from English to Japanese, back translation, expert review, and reconciliation. We then evaluated the reliability and validity of the Japanese version in a sample of 301 young women. RESULTS: The Japanese version of the Health-ITUES demonstrated high internal consistency (Cronbach α=0.85-0.98). Confirmatory factor analysis supported acceptable construct validity (root mean square error of approximation is 0.10, comparative fit index>0.90). Additionally, the Health-ITUES scores showed strong correlations with satisfaction and usage intention for the tool (r=0.779 and 0.797, respectively). CONCLUSIONS: The Japanese version of the Health-ITUES provides initial evidence of reliability and validity in an STI-related scenario among young women and may facilitate more rigorous usability evaluations of mHealth and conversational AI tools in Japan.

References

1. 1.Hato, T., Ishikawa, H., Todo, K., Harada, K., Yoshikawa, A., Fukuda, Y., & Schnall, R. (2026). Japanese Health Information Technology Usability Evaluation Scale for sexually transmitted infection-related chatbots: Development and psychometric validation study. JMIR Human Factors. https://doi.org/10.2196/90483