Predicting Momentary Suicidal Ideation From Smartphone Screenshots Using Vision-Language Models: Prospective Machine Learning Study

Clinical Snapshot

Key Findings

• P Value: Not reported for primary outcomes

• Effect Size: AUC=0.83 for temporal holdout, AUC≈0.50 for subject holdout

• Primary Outcome: Prediction of momentary suicidal ideation from smartphone screenshots

• Nnt Or Sensitivity: AUPRC=0.77 for temporal holdout, indicating good precision-recall balance

• Confidence Interval: 95% CI 0.003-0.07 for VLM vs text-only model difference

Clinical Application

Limited by poor generalization, privacy concerns, and need for extensive personalization period Would require TGA approval as medical device, raises significant privacy concerns under Australian privacy laws, not aligned with current RACGP suicide prevention guidelines Adults with established suicidal ideation requiring intensive monitoring

Abstract

BACKGROUND: Passive smartphone sensing shows promise for suicide prevention, but behavioral metadata (GPS, screen time, and accelerometry) often lacks the contextual information needed to detect acute psychological distress. Analyzing what people actually see, read, and type on their phones-rather than just usage patterns-may provide more proximal signals of risk. OBJECTIVE: This study aimed to test whether vision-language models (VLMs) applied to passively captured smartphone screenshots can predict momentary suicidal ideation (SI). METHODS: Seventy-nine adults with past month suicidal thoughts or behaviors completed ecological momentary assessments (EMA) over 28 days while screenshots were captured every 5 seconds during active phone use. We fine-tuned open-source VLMs (Qwen2.5-VL [Alibaba Cloud], LFM2-VL [Liquid AI]), and text-only models (Qwen3 [Alibaba Cloud]) to predict SI from screenshots captured in the 2 hours preceding each EMA. We evaluated performance with temporal and subject holdouts. RESULTS: The analytic sample comprised 2.5 million screenshots from 70 participants. Temporal holdout models achieved strong discrimination at the EMA level (AUC=0.83; AUPRC=0.77), with image-based models outperforming text-only models (AUC=0.83 vs 0.79; 95% CI 0.003-0.07). Subject holdout generalization was near chance (AUC≈0.50), though a simple lexical screening method retained modest discrimination (AUC=0.62). Smaller models performed comparably to larger models, supporting feasible on-device deployment. CONCLUSIONS: Screen content predicts short-term SI with clinically meaningful accuracy when models are personalized but does not generalize across individuals. These findings support a 2-stage clinical architecture, coarse lexical screening for new patients, with personalized VLM-based monitoring after a calibration period. On-device inference may enable privacy-preserving deployment.

References

1. 1.Jacobucci, R., Shao, W., Kobrinsky, V., & Ammerman, B. (2026). Predicting momentary suicidal ideation from smartphone screenshots using vision-language models: Prospective machine learning study. JMIR Mental Health. https://doi.org/10.1007/s12160-016-9830-8