Detection of cancer recurrence from Thai-English electronic medical records using sentence embeddings.

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract

• Effect Size: Development dataset — MetBERT: locoregional vs. no recurrence AUPRC 11.1%; distant vs. no recurrence AUPRC (monolingual-SBERT best) 32.0%; locoregional vs. distant recurrence AUPRC 91.7%. External validation — MetBERT: locoregional vs. no recurrence AUPRC 9.30%–21.50%; bilingual-SBERT: distant vs. no recurrence AUPRC 17.55%–24.39%; MetBERT: locoregional vs. distant recurrence AUPRC 88.30%–94.70%; bilingual-SBERT: locoregional vs. distant recurrence AUPRC 85.25%–91.80%

• Primary Outcome: Area under the precision-recall curve (AUPRC) for three cancer recurrence classification tasks in development and external validation datasets

• Nnt Or Sensitivity: Sensitivity and specificity at defined operating thresholds not reported; AUPRC used as summary metric. For locoregional vs. distant recurrence (the best-performing task), AUPRC 88.30%–94.70% (MetBERT) suggests reasonable discriminative ability. For recurrence vs. no recurrence tasks, AUPRC 9%–32% indicates poor-to-modest performance in the context of ~1% prevalence

• Confidence Interval: Not reported; inter-site ranges presented instead of formal 95% confidence intervals

Clinical Application

Technically feasible as a registry screening tool in institutions with structured EMR systems and NLP infrastructure. Bilingual-SBERT offers a computationally lighter alternative to MetBERT with competitive external validation performance. Text-length constraints of transformer models require document segmentation strategies. Clinical integration would require defined operating thresholds, human-in-the-loop review, and prospective validation before deployment. Direct applicability to Australian clinical practice is limited. Australian EMRs are predominantly English-language, removing the bilingual challenge that motivates this study. However, the methodological framework is relevant to: (1) Aboriginal and Torres Strait Islander health services where mixed-language or non-standard documentation occurs; (2) cancer registry automation within state-based registries (e.g., AIHW, state cancer registries); (3) RACGP and Cancer Australia interest in automated surveillance tools for cancer recurrence monitoring. The TGA would classify such a tool as a Software as a Medical Device (SaMD) requiring conformity assessment if used for clinical decision support. PBS implications are indirect. Australian cancer registries (e.g., Victorian Cancer Registry) may find the registry workflow automation concept relevant, though local validation on Australian EMR data would be mandatory before adoption. Patients with breast, colorectal, cervical, or head and neck cancer documented in mixed Thai-English EMR systems at Thai cancer hospitals; intended application is cancer registry workflow automation rather than individual patient clinical decision support

Abstract

OBJECTIVE: This study developed and validated monolingual and bilingual sentence-bidirectional encoder representations from transformers (SBERT) models for detecting cancer recurrence within Thai-English electronic medical records (EMRs) from Thai cancer hospitals. METHOD: A multicentre dataset of 32 436 documents from 1250 patients was used for model development. External validation involved an independent dataset of 9244 documents from 384 patients across two Thai cancer hospitals. Performance was benchmarked against a fine-tuned PubMedBERT (MetBERT). RESULTS: The development dataset included breast (43.9%), colorectal (12.1%), cervical (28.0%) and head and neck (16.0%) cancers. MetBERT achieved the highest area under the precision-recall curve (AUPRC) for locoregional versus no recurrence (11.1%) and locoregional versus distant recurrence (91.7%), while monolingual-SBERT excelled at distant versus no recurrence (32.0%). External validation demonstrated MetBERT superiority for locoregional versus no recurrence (9.30%-21.50%). For distant versus no recurrence, bilingual-SBERT performed best with AUPRC 17.55%-24.39%. While MetBERT led in distinguishing locoregional versus distant recurrence (88.30%-94.70%), bilingual-SBERT demonstrated robust external validation performance (AUPRC 85.25%-91.80%). DISCUSSION: Low AUPRC values (9%-32%) reflect the extreme class imbalance in real-world data (~1% recurrence prevalence). Despite this, fine-tuned MetBERT achieved highest performance, while bilingual-SBERT demonstrated superior robustness during external validation. This validates sentence embedding models for handling mixed Thai-English medical records in multilingual clinical environments. CONCLUSION: Sentence embedding frameworks provide a practical, generalisable solution for detecting cancer recurrence within multilingual EMRs. Despite text-length constraints, these models are suitable for clinical integration as a screening tool for cancer registry workflows.

References

1. 1.Sangariyavanich, E., Ponthongmak, W., Theera-Ampornpunt, N., Tangchitnob, N., McKay, G. J., & Thakkinstian, A. (2026). Detection of cancer recurrence from Thai-English electronic medical records using sentence embeddings. BMJ Health & Care Informatics. https://doi.org/10.1136/bmjhci-2025-101997