Automated phenotyping of ophthalmologic diseases from routine medical records using small language models and the human phenotype ontology (HPO)

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: F1 score of 0.80 indicating good overall performance

• Primary Outcome: Automated extraction accuracy of HPO terms from ophthalmology records

• Nnt Or Sensitivity: Precision 0.83, Recall 0.82, Jaccard similarity 0.67

• Confidence Interval: Not reported

Clinical Application

Requires local implementation of specialized language models and HPO integration, potentially challenging for smaller practices Could support Australian Digital Health initiatives and improve interoperability between ophthalmology services, though validation in Australian clinical settings needed Ophthalmology practices using narrative clinical documentation requiring standardized phenotype coding

Abstract

Automated phenotyping in ophthalmology requires accurate standardization of clinical terms to facilitate interoperability and research. This study evaluates the suitability of the human phenotype ontology (HPO) for automated extraction of ophthalmic phenotypes from narrative documentation. We developed a locally operated AI pipeline combining text segmentation and negation detection based on a small language model (PHI-4) with a dense retrieval approach using an augmented multilingual HPO catalog. Synonyms were incorporated into the HPO during training on anonymized consecutive physician letters. To validate the pipeline, 175 anterior segment and fundus descriptions from randomly picked medical records were manually annotated with HPO terms as ground truth. Overall, 342 HPO terms were identified manually (on average 2.53 terms per document), with 341 retrieved by the pipeline (on average 2.52 terms per document). Performance metrics showed a median Jaccard similarity of 0.67, precision of 0.83, recall of 0.82, and F1 score of 0.80. These results demonstrate that our AI pipeline effectively extracts standardized HPO terms from free-text ophthalmic findings. Integration of this pipeline into clinical information systems may enhance data interoperability and reduce manual coding workload in ophthalmology practices in the future.

References

1. 1.Thai, B. D., Arens, S., Reinhard, T., & Böhringer, D. (2024). Automated phenotyping of ophthalmologic diseases from routine medical records using small language models and the human phenotype ontology (HPO). Scientific Reports, 14, Article 59926-3. https://doi.org/10.1038/s41598-024-59926-3