Extracting user profile via large language models and ontologies
Clinical Snapshot
PICO Framework
| P — Population | Persian language dialogue systems and user interactions |
| I — Intervention | Hybrid method combining slot filling, in-context learning, and ontology-based inference for user profile extraction |
| C — Comparator | State-of-the-art large language models (GPT-4o, Llama-3-70B) with few-shot examples and fine-tuned versions |
| O — Outcomes | F-score for profile extraction accuracy, accuracy for inconsistency detection in user information |
Bottom Line
This computational study presents a hybrid approach for extracting user profiles from Persian dialogue systems, achieving substantial improvements over existing large language models. While the technical advances are notable, the clinical relevance is limited to healthcare applications involving Persian-speaking populations or multilingual patient interaction systems. The methodology lacks statistical rigor with no confidence intervals or significance testing reported. For Australian healthcare professionals, this work primarily offers insights into natural language processing techniques that could potentially be adapted for patient communication systems serving culturally and linguistically diverse communities. However, direct clinical application would require extensive validation in English-speaking contexts and consideration of privacy, implementation costs, and regulatory requirements under Australian healthcare frameworks.
Key Findings
P Value: Not reported
Effect Size: F-score of 90.46% for hybrid method
Primary Outcome: F-score for user profile extraction accuracy
Nnt Or Sensitivity: 17.08% absolute improvement over GPT-4o, 24.51% over Llama-3-70B
Confidence Interval: Not reported
Clinical Application
Requires significant computational infrastructure and Persian language expertise for implementation Minimal direct relevance to Australian healthcare practice, though methodology could inform development of multilingual patient interaction systems for diverse populations Limited direct clinical applicability - primarily relevant for healthcare chatbots or patient interaction systems in Persian-speaking populations
Abstract
Extracting user-specific profiles that include general personal information, such as hobbies, occupation, or age, is a valuable asset for systems like recommendation engines or personalized chatbots. Currently, most approaches moved toward exploiting the capabilities of large language models (LLMs) on rich languages while less-resourced languages still present significant opportunities for exploration and improvement. In this research, we present a multi-step approach for profile extraction from dialogue systems in Persian as a use case. Through an extensive set of experiments and analyses on various models, we show that LLMs struggle with these languages due to limited language-specific resources and complex linguistic structures. To address this, we propose a hybrid method that combines techniques such as slot filling, in-context learning, and ontology-based inference. Our final results demonstrate a significant improvement over current state-of-the-art models, including LLMs with few-shot examples and even their fine-tuned version. Our method achieves an F-score of 90.46, outperforming GPT-4o and Llama-3-70B by an absolute difference of 17.08 and 24.51 respectively. Our system can also detect inconsistencies in presented information in which our performance substantially exceeds the best performance of GPT-4o and Llama-3-70B with an accuracy of 92%. It is 21% absolutely better than GPT-4o and 47% better than Llama.
References
- 1.Safari, P., & Shamsfard, M. (2026). Extracting user profile via large language models and ontologies. PLoS One. https://doi.org/10.1371/journal.pone.0329934
Related Research
The Journal of rheumatology
Examining the Role of Wearables in Inflammatory Arthritis Care: A Narrative Literature Review
3 Aug 2026
Acta obstetricia et gynecologica Scandinavica
ChatGPT in urogynecology: Comparing large language model responses to human experts
3 Aug 2026
Sleep medicine
Concordance of wearable device sleep metrics with patient-reported sleep quality: A systematic review
3 Aug 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service