Large language models in public health: opportunity or threat? The case of button battery injuries.

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: ChatGPT-o1-preview: 8/10 answers rated accurate and complete vs ChatGPT-4o performance

• Primary Outcome: None of the LLM answers were rated as completely or partially incorrect

• Nnt Or Sensitivity: 100% of responses rated as at least partially correct

• Confidence Interval: Not reported

Clinical Application

Preliminary evidence suggests feasibility but requires validation against clinical guidelines and expert consensus Relevant to Australian paediatric emergency departments and poison information centres; would need validation against RACGP and APLS guidelines for button battery management Public health professionals and paediatric healthcare providers managing button battery exposures

Abstract

BACKGROUND: Button battery (BB) injuries in children represent a severe and growing public health burden. The literature on the topic is extensive; however, there is a notable lack of structured public health initiatives addressing the problem. The present study aimed to test the feasibility of using large language models (LLMs) to draft recommendations for preventing and managing BB ingestion in children. METHODS: A set of questions was generated and submitted to ChatGPT-4o and ChatGPT-o1-preview. Questions were based on statements and websites of scientific societies and not-for-profit organisations and were developed to produce comprehensive recommendations that provided information on BB risks, primary and secondary prevention, clinical management and follow-up, and general public health initiatives. Two independent reviewers rated the accuracy and readability of the questions submitted to the LLMs. The accuracy was rated using a four-level scale, while the readability was assessed using two established readability tools, the Flesch Reading Ease (FRE) and the Flesch-Kincaid Grade Level (FKGL). RESULTS: None of the answers provided by the LLMs were rated as completely incorrect or partially incorrect. ChatGPT-o1-preview outperformed ChatGPT-4o in accuracy, with eight answers rated as accurate and complete. Both models showed similar readability levels, with high FKGL and FRE scores indicating college-level comprehension. DISCUSSION: LLM demonstrated a strong performance in this study, with no responses rated as incorrect or partially incorrect, showing its great potential and feasibility for use in public health. CONCLUSIONS: The present findings suggested the potential feasibility of LLMs in public health for preventing paediatric injuries.

References

1. 1.Lorenzoni, G., & Gregori, D. (2026). Large language models in public health: opportunity or threat? The case of button battery injuries. Injury Prevention. https://doi.org/10.1136/ip-2025-045655