Development and Psychometric Validation of a Usability Instrument Based on ISO 25010 for Electronic Health Record Systems in Peruvian Health Care Settings: Methodological Study

Clinical Snapshot

Key Findings

• P Value: Not reported for primary validation outcomes

• Effect Size: High content validity (Aiken's V=0.95) and excellent internal consistency (α=0.968)

• Primary Outcome: Instrument validation with excellent psychometric properties

• Nnt Or Sensitivity: Sensitivity: 81.7% rated usability as very high, 18.3% as high; Specificity: 80% compliance with regulatory standards

• Confidence Interval: 95% CI 0.84-0.98 for Aiken's V coefficient

Clinical Application

Moderate - requires trained personnel for administration and interpretation of psychometric results Limited direct applicability due to different regulatory framework, but methodology could inform development of similar instruments for Australian Digital Health Standards Healthcare organisations implementing or evaluating EHR systems, particularly in Spanish-speaking contexts

Abstract

BACKGROUND: The usability of electronic health records influences patient safety, clinical efficiency, and the sustainable digital transformation of health care. While tools such as the System Usability Scale exist, few are based on structured software quality models or align with local regulatory frameworks, particularly in low- and middle-income countries. OBJECTIVE: This study aims to develop and psychometrically validate an instrument to assess electronic health record usability, based on the ISO 25010 (International Organization for Standardization) quality model, and in accordance with the Peruvian Technical Health Standard (Norma Técnica de Salud [NTS]). METHODS: A 3-stage methodological study was conducted following COSMIN (Consensus-based Standards for the Selection of Health Measurement Instruments) guidelines. First, a 23-item instrument was developed, grouped into 7 usability dimensions based on ISO 25010. Second, content validity was assessed using a modified Delphi method with 10 experts, using Aiken's V coefficient. Third, structural validity and reliability were evaluated in 115 participants (physicians, nurses, and administrative staff) using confirmatory factor analysis and Cronbach α. Additionally, the module's functional compliance with the NTS was verified. RESULTS: The instrument demonstrated excellent psychometric properties among 115 participants (53 physicians, 48 nurses, and 14 administrative staff); high content validity (Aiken's V=0.95; 95% CI 0.84-0.98), excellent internal consistency (α=.968), and optimal factor fit (Comparative Fit Index=1.000; Tucker-Lewis Index=1.001). Overall usability was perceived as very high by 94 of 115 users (81.7%), and high by 21 of 115 users (18.3%), with no perceptions at lower levels. The module met 28 of 35 requirements (80%) of the Peruvian Technical Health Standard. Administrative staff reported higher satisfaction (13/14, 93% rated it as very high) compared with clinical professionals (81/101, 80% rated it as very high), particularly in the dimensions of error protection and operability. CONCLUSIONS: This study provides the first fully psychometrically validated instrument in Spanish based on ISO 25010, System Usability Scale, and other theoretical foundations for evaluating the usability of information systems. The results highlight the importance of considering both international standards and local regulatory requirements (NTS) in the design of digital health systems. The instrument is applicable in Spanish-speaking contexts and can serve as a reference for future usability evaluations in the region as well as in low- and middle-income countries.

References

1. 1.Mamani-Pari, D., Peñaloza-Hualpa, A. F., Humpiri-Flores, M. E., Gutierrez-Quispe, E., & Otazu-Luque, J. E. (2026). Development and psychometric validation of a usability instrument based on ISO 25010 for electronic health record systems in Peruvian health care settings: Methodological study. JMIR Human Factors. https://doi.org/10.2196/81377