Can NLP Detect Loneliness in Electronic Health Records? A Proof-of-Concept Study

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Accuracy range 0.3 to 0.7 across different NLP approaches

• Primary Outcome: NLP model accuracy for loneliness severity classification

• Nnt Or Sensitivity: Sensitivity and specificity not reported; models struggled particularly with severe loneliness detection

• Confidence Interval: Not reported

Clinical Application

Low feasibility given modest accuracy and requirement for specialized NLP infrastructure Limited relevance to Australian practice given poor performance; would not meet TGA standards for clinical decision support tools. RACGP guidelines emphasize social determinants screening, but this approach appears inadequate for reliable detection Limited applicability due to poor performance and sparse documentation in routine clinical practice

Abstract

Loneliness is clinically important but under-documented in electronic health records (EHRs), posing challenges for secondary use and computational phenotyping. This study evaluated whether natural language processing (NLP) methods can detect and classify loneliness severity from clinical notes. Patients with a loneliness survey (mild, moderate, severe) were identified, and notes within six months prior to the survey were retrieved. An expert-expanded lexicon was applied, and transformer models (RoBERTa, ClinicalBERT, Longformer) were fine-tuned for loneliness severity classification. Large language model-based summarization of social and psychiatric history was also tested as an alternative input representation. Performance was evaluated using accuracy, weighted-F1, and per-class F1. All models achieved modest accuracy (0.3 to 0.7), and struggled to identify severe loneliness, reflecting sparse and inconsistent documentation even among surveyed patients. While summarization marginally improved accuracy, gains primarily reflected mild predictions. Manual review of 100 social worker notes from severely lonely patients found explicit mentions of loneliness in only two cases, confirming that relevant documentation is exceedingly rare. These findings demonstrate that model performance is constrained by the sparse and inconsistent documentation of loneliness in EHRs, rather than by deficiencies in the modeling approach itself.