Disparate language and model effects on AI-based translation and recognition of genetic conditions.

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract; statistical significance claimed without quantification

• Effect Size: Not quantified in the abstract; described as 'significant differences' based on translation method, condition, language, and prediction model

• Primary Outcome: Accuracy of LLM-based identification of 40 genetic conditions from AI-translated descriptions across up to 139 (NMT) and 70 (LLM) languages

• Nnt Or Sensitivity: Sensitivity/specificity or accuracy metrics not reported in abstract; adaptive translation did not improve accuracy with 3 tested LLMs but showed benefit with smaller LLMs in one language

• Confidence Interval: Not reported

Clinical Application

Findings are directly applicable to clinical informatics and health technology assessment contexts. Clinicians and health systems deploying AI translation tools for genetic medicine should be aware of language-dependent performance variability. Adaptive translation strategies require further validation before routine clinical implementation. Australia's population includes over 300 spoken languages, with approximately 21% of Australians speaking a language other than English at home (ABS 2021). The RACGP and ACMG both emphasise culturally and linguistically appropriate care. Genetic services in Australia, including those funded through Medicare and state-based health services, increasingly use digital and telehealth platforms. The TGA does not currently regulate LLM-based translation tools as medical devices, creating a regulatory gap. PBS-listed genetic testing pathways and the Australian Genomics Health Alliance's equity frameworks are directly relevant. This study underscores the risk that AI-assisted genetic counselling tools may perform inequitably for CALD (Culturally and Linguistically Diverse) patients — a priority population in Australian health policy. Patients and clinicians using AI-based tools for genetic condition information in non-English languages; particularly relevant for multilingual genetic counselling, rare disease diagnosis support, and patient-facing health information platforms

Abstract

INTRODUCTION: Artificial intelligence (AI) is increasingly prevalent. Patients and clinicians may use AI-based tools in many different languages. OBJECTIVE: To investigate AI translation tools for descriptions of genetic conditions and how AI identification of genetic conditions is affected by translations. MATERIALS AND METHODS: We used Neural machine translation (NMT) and large language-model (LLM) translation to translate descriptions of 40 genetic conditions into 191 and 93 languages, respectively. Excluding translations retaining English medical terms verbatim, we respectively focused on 139 and 70 languages. After assessing translations, we assessed the ability of 3 proprietary and 3 open-weight general LLMs to identify conditions in the translations. We analyzed how accuracy was affected by the conditions' prevalence in the literature, and attributes of the languages (the script, language family, and prevalence of the language in training sources). We also investigated adaptive translation for select languages. RESULTS: We found significant differences in condition identification based on the translation method, condition, language, and prediction model. The accuracy of some models was more affected than others by factors like the conditions' literature prevalence, language script, family, and language prevalence. Adaptive translation for select languages did not improve translations or diagnostic accuracy with the 3 tested LLMs. However, further analysis with 1 language showed that this approach was more effective with smaller LLMs. CONCLUSIONS: AI-based translation has variable performance, which can affect the ability of AI models to recognize genetic conditions. These findings should inform safe medical AI use to support consistent performance in different languages.

References

1. 1.Duong, D., Manoli, I., Phadke, S. R., Phornphutkul, C., Raymond, J. D., & Solomon, B. D. (2026). Disparate language and model effects on AI-based translation and recognition of genetic conditions. Journal of the American Medical Informatics Association: JAMIA. https://doi.org/10.1371/journal.pgen.1011168