Research Appraisals
Evidence-based critical appraisals of the latest medical research, systematically evaluated using Oxford CEBM methodology.
Showing 26 appraisals
Scientific reports
The role of AI in combating misinformation: leveraging text mining and social networking analysis
Misinformation on social media can be a severe threat to social trust, safety, and health of the population, especially in times of an epidemic like the Monkeypox outbreak. This study specifically focuses on a hybrid RoBERTa-GRU architecture designed to capture both contextual semantics and temporal dependencies in social media discourse. This research presents how the combination of text mining and social network analysis enables Artificial Intelligence (AI) to support misinformation detection. The proposal of a hybrid architecture that integrates RoBERTa and GRU-based embeddings in a contextual fashion and GRU-based modelling of sequential patterns helps identify and substantiate misinformation in social media posts. Based on a curated X (formerly Twitter) dataset consisting of Monkeypox posts (5787 posts), the model provided state-of-the-art results, with ROC-AUC 0.9979 and Cohen's kappa 0.9887; standalone baselines were also surpassed. Results reveal that the proposed transformer-RNN hybrid effectively captures both semantic depth and temporal relationships in misinformation detection tasks. In addition to performance, the paper addresses the limitations of dataset bias, multilingual constraint issues, and scalability as related to cross-linguistic applicability, multimodal study, and performance in real-time and resource-limited systems. The study adds value in this emerging body of knowledge on AI-driven social media analytics by offering practical guidance for mitigating health-related misinformation online.
22 July 2026
Read appraisal →PloS one
Self-positioning in space science communication: A corpus-assisted discourse study
This study incorporates text mining into critical discourse analysis to examine how government science agencies in China and the United States position themselves in space science communication on social media. Two specialized corpora have been built by collecting posts from government science agencies on Weibo from China and X (formerly Twitter) from the U.S. With the help of the text mining tool KH Coder, this study gives a corpus-assisted discourse analysis of the particular ways of positioning at different levels of discourse: (1) topics/themes, (2) addressing terms, and (3) those words that co-occur with self-addressing terms in a sentence. The findings reveal significant differences in their preferential ways of positioning. Chinese government science agencies present themselves as state-affiliated yet approachable institutions, blending achievements and operational efficiency with patriotism and collective pride. Their use of diverse addressing terms and co-occurrence patterns portrays them as experienced, supportive guides, balancing national identity with interpersonal closeness. In contrast, U.S. government science agencies emphasize professionalism, focusing on research, space exploration, mission execution, and audience engagement, with minimal reference to state affiliation. Their addressing terms are formal and standardized, with pride centred on mission success and discovery, highlighting expertise and scientific leadership. Their preferential ways of positioning are further explained in their respective contexts in order to present a proper understanding of these differences.
12 July 2026
Read appraisal →PloS one
Transformer-based advertisement image layout generation and object fidelity optimization
The layout of the image and the positional distribution of the objects directly affect the audience's visual focus and the effect of the message conveyed. By setting up a specific layout structure, the viewer's attention can be more focused on the product launched or the moral of the advertisement. Existing work related to position-controllable text-to-image generation has made great progress in generating results on simple images. However, when generating images of complex scenes, the quality is often poor. This can result in the model failing to accurately convey the message of the advertisement when used to generate advertisements. To address these limitations, we propose LAYOBJ-GAN, a novel two-stage framework for layout-controllable advertisement image synthesis. Unlike prior work, our method explicitly models background layouts jointly with object layouts during the text-to-layout generation stage, enabling comprehensive spatial planning of complex scenes. Technically, we introduce a Transformer-based sequence-to-sequence layout generator that learns long-range dependencies between textual descriptions and both background and object regions, which has not been explored in previous advertisement-oriented text-to-image frameworks. In the layout-to-image stage, we further propose a fine-grained text-layout interaction normalization module (TL-Norm) that enables object knowledge transfer from a pre-trained category-to-image model, allowing object appearance to be adaptively modulated by textual context and layout constraints. Extensive experiments on MS-COCO and a high-definition advertisement dataset (AsHQ-10K) demonstrate that LAYOBJ-GAN significantly outperforms seven state-of-the-art methods in image quality, layout controllability, and semantic object accuracy. These results confirm the effectiveness of explicitly modeling background layouts and transferring object-level generative knowledge for complex advertisement image synthesis.
27 June 2026
Read appraisal →Journal of Korean medical science
Evaluating Large Language Models for Post-Publication Promotion: A Blinded Comparative Study of Social Media Posts in Public Health
BACKGROUND: Social media platforms such as X (formerly Twitter) are increasingly used by journals, authors, and institutions to promote newly published research. Well-designed posts can enhance visibility, accelerate knowledge translation, and increase altmetric attention. However, creating accurate and policy-compliant content is time-intensive. Large language models (LLMs) offer a potential solution, yet systematic evaluations of their performance in post-publication promotion remain limited. METHODS: We conducted a blinded, crossed, offline evaluation of four LLMs: GPT-5 (OpenAI), Gemini 2.5 Pro (Google DeepMind), Grok-3 (xAI), and Perplexity Pro (Perplexity AI), tasked with generating X-style posts (≤ 260 characters) for 36 open access articles from The Lancet Public Health, The Lancet Planetary Health, and Annual Review of Public Health. Posts were generated using a standardized system and user prompt. A single blinded rater scored outputs using a five-domain rubric (factual accuracy, clarity, policy compliance, call-to-action quality, structure/metadata; maximum score 10). Secondary measures included character count, hashtag use, and readability (Flesch-Kincaid Grade Level). General linear models with Bonferroni-adjusted post hoc tests and non-parametric analyses were applied. RESULTS: All four models achieved perfect factual accuracy and no policy violations. Mean total quality scores differed significantly by model, P < 0.001. GPT-5 (9.60) and Perplexity Pro (9.60) performed best, followed by Gemini 2.5 Pro (9.47), while Grok-3 scored lower (8.80). Domain analyses showed Grok-3 underperformed in call-to-action quality (1.40 vs. ≥1.97 in other models, P < 0.001) and produced significantly shorter posts (median 194 characters, P < 0.001). Perplexity Pro scored highest for policy compliance, while GPT-5 and Gemini 2.5 Pro achieved superior structural scores. Readability varied: GPT-5 8.9 (7.3-9.2) and Perplexity Pro 7.3 (6.5-8.8) generated more complex outputs, whereas Gemini 2.5 Pro 5.1 (4.8-6.5) and Grok-3 4.5 (3.6-6.3) produced more accessible posts. CONCLUSION: LLMs can reliably generate accurate and policy-compliant social media posts for research promotion, with differences in style and readability that may inform audience targeting. GPT-5, Gemini 2.5 Pro, and Perplexity Pro produced high-quality outputs, while Grok-3 underperformed across several domains. These findings highlight the potential of LLMs as scalable first-draft tools for post-publication promotion, capable of improving the reach and accessibility of scientific research. Careful model selection, tailored to audience and communication goals, together with human oversight, remains essential.
25 June 2026
Read appraisal →Journal of medical Internet research
Trusting Generative AI for Health Advice: Preregistered Survey Experiment
BACKGROUND: Generative artificial intelligence (AI) systems are increasingly used for health information seeking, yet it remains unclear how the public evaluates AI-generated health advice relative to guidance from credentialed clinicians in digital environments. Understanding the conditions under which AI is perceived as credible is critical as these systems become integrated into digital health ecosystems. OBJECTIVE: This study examined how source type (a human nurse in an online portal, a health care-specialized generative artificial intelligence system framed as an "AI nurse," or ChatGPT [OpenAI], a general-purpose chatbot), message characteristics, contextual risk, values framing, and individual differences in medical skepticism and experience with AI shape credibility evaluations of the provided advice and its purported source. METHODS: In a preregistered online experiment, a national sample of US adult participants (N=1502) was randomly assigned to 1 of 3 source conditions and evaluated health advice across 3 scenarios: low risk (dietary advice for cholesterol), high risk (chest pain triage), and a morally sensitive scenario (egg freezing). Advice type (intuitive vs counterintuitive) was manipulated in the risk scenarios, and ideological framing (neutral, conservative-leaning, and liberal-leaning) was manipulated in the morally sensitive scenario. Primary outcomes included participants' perceived credibility of the advice and beliefs about whether the patient should follow it. Source-level perceptions of competence and benevolence were also assessed. Medical skepticism and prior AI experience were examined as moderators. RESULTS: Advice attributed to a human nurse was rated as more credible than advice attributed to either AI source. Message intuitiveness showed effects comparable to, and sometimes larger than, the effects of source: intuitive advice was perceived as more credible than counterintuitive advice, with this difference amplified in high-risk contexts. In the morally sensitive scenario, ideological framing influenced perceived bias but did not interact significantly with source. Medical skepticism moderated source evaluations: higher skepticism was associated with greater perceived competence of the AI nurse and lower perceived competence of the human nurse. CONCLUSIONS: Generative AI is evaluated within existing credibility frameworks rather than dismissed outright as inferior to human expertise. While licensed clinicians retain a credibility advantage, AI-generated advice is generally perceived as competent and legitimate. Importantly, individuals skeptical of traditional medical authority may evaluate AI-based guidance more favorably, suggesting that AI systems may redistribute-rather than uniformly erode-trust in health advice. As AI tools become embedded in patient-facing health platforms, message design and audience characteristics may shape acceptance more strongly than source labeling alone.
24 June 2026
Read appraisal →Cancer
Alcohol consumers' receptivity to artificial intelligence-generated alcohol-cancer risk messages: An experimental study.
BACKGROUND: Alcohol is a group 1 carcinogen linked to seven cancers, yet awareness of this risk remains low in the United State. Identifying effective alcohol-cancer communication strategies is a public health priority. The objective of this study was to test the effects of message specificity (general vs. cancer-specific) and visual intensity (text-only, neutral pictorial, graphic pictorial) on message receptivity (attention, emotional reactions) and precursors of behavior (harm perception; intentions to reduce, limit, or stop drinking) among moderate and heavy alcohol consumers. METHODS: Eight evidence-based text messages were created across two topics: general and cancer-specific harm. Artificial intelligence was used to generate pictorial versions at two intensities: neutral (symbolic, no visible disease) and graphic (explicit health consequences). In a 2025 online within-subject/between-subject crossover experiment, 639 US adult consumers (aged 21 years and older; 49.3% female) each viewed two randomly selected messages (one general, one cancer-specific) presented in three formats (text-only, neutral, and graphic; for six total formats), with counterbalanced order. Linear mixed-effects models were used to estimate differences by intensity, specificity, and drinking levels, reporting regression coefficients (β, 95% confidence intervals). RESULTS: Cancer-specific messages produced greater attention, emotional reactions, and intentions to reduce drinking than general messages (β = 0.10-0.19; p < .05). Graphic pictorials outperformed neutral images on emotional and behavioral outcomes (β = 0.19-0.22; p < .05). Moderate consumers showed stronger perceived harm and message responsiveness than heavy consumers (β = 0.29-0.77; p < .05). CONCLUSIONS: Artificial intelligence-generated alcohol-cancer messages are feasible and effective in strengthening precursors to behavior change. Cancer-specific content and higher visual intensity enhance impact, particularly among moderate consumers, highlighting the importance of tailoring alcohol-cancer communication strategies to different audience characteristics. Many people in the United States are unaware that drinking alcohol increases the risk of several cancers. This study tested different ways of communicating the alcohol–cancer link, including text‐only messages, messages with artificial intelligence–generated images (neutral or graphic), and general versus cancer‐specific content; in addition, the authors examined whether responses differed between moderate drinkers and heavy drinkers. The authors observed that messages with images were more effective than those with text alone, especially when the images clearly showed health harm. Messages that named specific cancers (such as breast or colon cancer) were also more effective than general statements about cancer; however, heavy drinkers, who face the highest cancer risk, were less responsive overall. The findings suggest that effective alcohol–cancer communication should carefully consider what the message says, how it is presented, and who the audience is. Nonstigmatizing, harm‐reduction messages that encourage small, realistic changes and are delivered in supportive settings, such as conversations with health care providers, may work better for heavy drinkers. The study also highlights the promise of artificial intelligence as a tool for developing and tailoring health messages. Future research should explore how different messages influence behavior over time.
15 June 2026
Read appraisal →Journal of medical Internet research
Characteristics of Human-Like Virtual Profiles in Relation to Audience Reach and Engagement on Instagram: Secondary Data Analysis
BACKGROUND: Prior research on human-like virtual profiles (VPs)-computer-generated imagery (CGI; displaying legible artificiality) or artificial intelligence (AI)-generated personas on social media-is often based on small samples and limited data. Furthermore, prior studies group highly photorealistic virtual influencers with abstract virtual characters, complicating comparisons. Recent generative AI has enabled large-scale production of synthetic content. These advances have drastically increased the quality and volume of synthetic media. Much of the prior literature predates these developments, leaving open questions about how posting behavior and VP design now relate to reach and engagement outcomes. OBJECTIVE: This study aimed to examine how posting behavior and key VP attributes, namely, photorealism; physical, behavioral, and narrative consistency; and human copresence related to reach and engagement across content formats (static images vs videos) on Instagram. METHODS: A total of 157 human-like female VPs were included in the final Instagram dataset. During the initial screening stage, 5 human-like male macro and mega VPs were identified but excluded because their number was too small to support meaningful subgroup comparison. Engagement was operationalized as like rate (likes or followers) for images and videos; reach was measured via examined absolute impressions. Performance was summarized using the single best-performing post and the arithmetic mean of the top 3 posts per format. Profiles and posts were further coded using predefined content variables covering visual realism, identity consistency, copresence structure, appearance patterns, and body-type representation. RESULTS: Higher-performing content was more commonly observed among larger profiles, particularly for video metrics. Among videos exceeding 20 million impressions, out of 12 videos, 7 (58%) were produced by CGI-like VPs, whereas high video like rates were more often observed among photorealistic or face-swapped profiles. In the image engagement analysis, out of 8 highest-like-rate posts, 6 (75%) featured VPs with dark hair and dark eyes, and out of the 8 posts, 4 (50%) included copresence of multiple subjects. Furthermore, of the 157 top-performing profiles reviewed, 122 (77.71%) demonstrated stable visual identity and consistent behavioral and narrative presentation, whereas noticeable inconsistencies were rarely observed in this subset. These findings represent descriptive patterns within established macro- and mega-level profiles rather than as causal predictors of growth. CONCLUSIONS: Among established human-like female VPs on Instagram, higher engagement and reach frequently co-occurred with larger profile scale, identity coherence, and transparent virtual presentation than with photorealism alone. CGI-like aesthetics aligned more with algorithmic distribution, whereas photorealistic motion content featured more prominently among posts with high active engagement. These results suggest that audiences may respond more favorably to coherent and legible virtual identities than to ambiguous realism. Because the study was restricted to macro- and mega-level profiles, the observed traits should be interpreted as characteristics of successful incumbents rather than as general predictors of VP growth.
7 June 2026
Read appraisal →African journal of reproductive health
Artificial intelligence-enabled digital reproductive health education: corpus-based strategies for strengthening Spanish health communication capacity
This study examines how corpus-grounded artificial intelligence (AI) can strengthen Spanish reproductive health communication capacity within China's digital health ecosystem. A mixed-methods design was employed, combining corpus linguistics, AI-assisted message generation, expert-informed evaluation, and quantitative user assessment. A domain-specific Spanish reproductive health corpus was constructed from 3,052 documents, including clinical guidelines, patient education materials, and FAQs/user queries, yielding 1,065,110 tokens for linguistic analysis and AI grounding. Corpus-derived readability benchmarks, lexical simplification rules, and discourse patterns were integrated into an AI content generation pipeline to produce reproductive health messages, which were then compared with non-corpus-grounded AI outputs. The user evaluation phase was conducted among 240 Spanish-speaking or Spanish-proficient adults in selected Chinese metropolitan cities. Data were collected through a structured questionnaire measuring Corpus-Grounded AI Generation, Spanish Communication Quality, and Health Communication Capacity, and were analyzed using reliability testing, correlation analysis, and structural equation modeling. The findings showed that corpus-grounded AI significantly improved Spanish communication quality, while communication quality had the strongest effect on users' comprehension, confidence, and help-seeking intention. Mediation analysis further demonstrated that Spanish communication quality significantly mediated the relationship between corpus-grounded AI generation and health communication capacity. The study concludes that linguistically informed AI design can enhance the clarity, accessibility, and effectiveness of reproductive health education and offers a practical framework for multilingual digital health communication in sensitive healthcare contexts. Cette étude visait à examiner le rôle de l'IA guidée par la linguistique de corpus dans l'amélioration de la qualité de la communication en espagnol sur la santé reproductive au sein de l'écosystème numérique chinois. En utilisant une approche à méthodes mixtes, la recherche a combiné l'analyse de corpus de textes espagnols sur la santé reproductive, la génération de contenu basée sur l'IA et l'évaluation par les utilisateurs. Un corpus espagnol spécifique au domaine de la santé reproductive a été construit et intégré dans un pipeline de génération d'IA ancré dans le corpus pour optimiser la lisibilité, l'exactitude et la sensibilité culturelle. La qualité de la communication et la capacité de communication en santé ont été mesurées à l'aide d'une analyse de fiabilité, de tests de corrélation et de modélisation par équations structurelles. Les résultats ont révélé que l'IA ancrée dans le corpus améliore significativement la qualité de la communication en espagnol, ce qui améliore à son tour la compréhension des utilisateurs, l'auto-efficacité et les intentions de recherche d'aide. La qualité de la communication est apparue comme un mécanisme médiateur clé entre la conception de l'IA et les résultats en matière de communication santé. Sur le plan théorique, l'étude établit une base pour intégrer la linguistique de corpus dans les systèmes d'éducation à la santé pilotés par l'IA, tandis que sur le plan pratique, elle propose un cadre évolutif pour les plateformes multilingues d'éducation à la santé reproductive. Ces résultats ont des implications importantes pour les politiques mondiales de santé numérique, la gouvernance de l'IA et les stratégies de communication en santé reproductive.
30 May 2026
Read appraisal →JMIR perioperative medicine
AI-Generated Avatar Videos for Postoperative Patient Education Among Health Care Workers: Pilot Randomized Controlled Trial
BACKGROUND: Effective postoperative communication is vital for patient recovery, yet traditional text-based discharge instructions often lead to poor comprehension and adherence, particularly among patients with limited health literacy. Although educational videos improve understanding and retention, their widespread use has been hampered by high production costs. Generative artificial intelligence (AI) offers a scalable solution for creating engaging video content. OBJECTIVE: The primary objective of this pilot study was to assess the feasibility of creating and deploying AI-generated, avatar-led videos for postoperative instruction delivery. Secondary objectives included comparing knowledge retention, engagement, perceived clarity, and user experience between AI-generated video and traditional text-based handout formats among health care workers. METHODS: In this randomized pilot study, 38 health care worker volunteers were recruited as a convenience sample to pilot-test the intervention before patient implementation. Participants were assigned to either a text handout group (n=19, 50%) or an AI-generated video group (n=19, 50%). Both groups received information on 10 common postoperative topics. The primary outcome was objective knowledge, assessed via a 10-item quiz. Secondary outcomes, measured through surveys with 5-point Likert scales, included engagement time, subjective engagement, perceived clarity, usefulness, confidence in understanding, and information retention. Qualitative feedback was also collected. RESULTS: Objective knowledge quiz scores did not differ significantly between groups (mean 8.89, SD 1.20 for the AI-generated video group vs mean 8.21, SD 1.78 for the text handout group; P=.17; Cohen d=0.45). Participants in the AI-generated video group demonstrated significantly higher engagement time (mean 15.11, SD 7.78 minutes vs mean 8.84, SD 4.03 minutes; P=.004; Cohen d=1.04). They also rated instructions as significantly clearer (mean 4.63, SD 0.50 vs mean 4.00, SD 0.82; P=.007; Cohen d=0.93), more engaging (mean 4.05, SD 0.78 vs mean 3.32, SD 1.00; P=.02; Cohen d=0.81), and more effective for retention (mean 4.42, SD 0.84 vs mean 3.37, SD 0.68; P<.001; Cohen d=1.38). Qualitative feedback highlighted the engaging nature of AI-generated videos but noted areas for avatar refinement. CONCLUSIONS: In this pilot study with health care workers, AI-generated avatar videos did not improve objective knowledge scores but significantly enhanced engagement, perceived retention and perceived clarity (Cohen d=0.81-1.38). Future studies in actual patient populations with diverse health literacy levels are needed to determine whether these engagement advantages translate into improved knowledge outcomes.
28 May 2026
Read appraisal →JMIR formative research
Differences in Safety Risks Across Languages in Health-Relevant Queries: Vulnerability Analysis of Large Language Model Responses
BACKGROUND: Large language models (LLMs) such as ChatGPT are increasingly used to support health-related queries and decision-making. However, these models can be "jailbroken" through adversarial prompts that bypass safety filters and elicit harmful or medically inappropriate responses. In health care contexts, such vulnerabilities pose serious risks. Understanding how jailbreak susceptibility varies across languages is essential for developing robust safeguards and promoting equitable access to safe health information. This paper may contain examples that may be deemed harmful in terms of violence, self-harm, and drug abuse. OBJECTIVE: This study aims to systematically compare and contrast the vulnerability of a health LLM for jailbreaking across 3 languages: English, Spanish, and Hindi (transliterated using the Latin alphabet), based on emoji and permutation cipher attacks. METHODS: We analyzed 1000 input prompts per language, drawn from the BeaverTails dataset, across 3 harm categories: self-harm, violence, and drug abuse. Each prompt was modified using emoji and permutation cipher techniques, resulting in 6000 input-output pairs. Model responses were evaluated by human coders to determine the success rate of jailbreak attempts across languages and cipher types. RESULTS: Hindi prompts showed the highest vulnerability, with 787 successful jailbreaks using emoji ciphers and 873 using permutation ciphers. Spanish and English followed, with lower success rates across both cipher types. Differences in jailbreak success across languages and cipher strategies were statistically significant. Additionally, attacks targeting violence-related prompts were more successful overall than those targeting drug-related or self-harm content, indicating variation in vulnerability by harm type. CONCLUSIONS: The findings of this formative study reveal that LLM safety performance varies substantially across languages and harm categories, raising concerns about equitable protection in multilingual health communication. Disparities in access to harmful content may contribute to downstream health risks. Strengthening multilingual content moderation and developing language-aware safety mechanisms are critical steps toward creating safer and more inclusive health AI systems.
28 May 2026
Read appraisal →Journal of medical Internet research
Large Language Model-Generated Patient Instructions for Prescriptions in Primary Health Care: Preclinical Algorithm Validation
BACKGROUND: The application of generative artificial intelligence to simplify medication use instructions has the potential to enhance people's health by improving treatment adherence. OBJECTIVE: We evaluated the performance of large language models (LLMs) in generating medication usage instructions to complement prescriptions in primary health care. METHODS: This randomized, blinded experimental preclinical study used prescription-inducing scenarios, assigned to 62 health care professionals, to validate instructions generated by LLMs during electronic prescriptions. The instructions were generated by ChatGPT-4.0 (OpenAI), Llama3.1-8B (Meta), and Llama3.1-8B-RAG (Meta) using retrieval-augmented generation based on patient information leaflets. Performance metrics assessed adequacy, completeness, clarity, language simplification, usefulness, and errors in the generated instructions, with scores to analyze overall and individual metrics. RESULTS: The 3 models yielded high overall scores for producing qualified instructions (ChatGPT-4.0: median 88.4, IQR 22.8; Llama3.1-8B: median 66.5, IQR 50.9; Llama3.1-8B-RAG: median 79.9, IQR 34.4; Kruskal-Wallis test P=.003). Llama3.1-8B-RAG received evaluations with similar overall scores to ChatGPT-4.0 (post hoc test, P=.05) and similar to Llama3.1-8B (post hoc test, P=.44). ChatGPT-4.0 outperformed Llama3.1-8B (Bonferroni test, P<.001). Regarding specific domains, Llama3.1-8B-RAG received scores equivalent to those of ChatGPT-4.0 for adequacy (mean 6.24, SD 2.3 vs mean 6.82, SD 2.1; post hoc test, P=.54); completeness (mean 5.94, SD 2.2 vs 6.55, SD 1.9; post hoc test P=.38), clarity (mean 5.77, SD 2.4 vs mean 6.68, SD 1.9; post hoc test P=.09), and usefulness (mean 5.42, SD 2.4 vs mean 5.96, SD 2.2; post hoc test P=.63). ChatGPT-4.0 received higher scores in the language simplification criterion than Llama3.1-8B-RAG (mean 7.05, SD 1.5 vs mean 5.44, SD 2.6; post hoc test P<.001). Interrater variability in assigning scores ranged from 4.2% (n=3) to 85.8% (n=6) among primary health care professionals. Instructions leading to incorrect use of the medication had similar frequency among the models(ChatGPT-4.0: n=15, 22.7%; Llama3.1-8B: n=19, 22.8%; Llama3.1-8B-RAG: n=19, 22.8%; chi-square test P=.71). The frequencies of hallucination were similar (ChatGPT-4.0: n=7, 10.6%; Llama3.1-8B: n=9, 13.6%; Llama3.1-8B-RAG: n=6, 9.1%; chi-square test P=.67). CONCLUSIONS: The open-source LLM enhanced with external information presented similar performance to the closed-source model, except for ChatGPT4.0, which was superior in language simplification of messages. LLM generation demonstrated potential for instructing patients on medication use. Nonetheless, the introduction of this innovation into the electronic prescribing workflow demands prescriber validation for human oversight of the technology and requires a strategy for LLM performance governance. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.12688/verixiv.1359.1.
27 May 2026
Read appraisal →Journal of medical Internet research
Receipt of Medicines Information From the Internet and Other Information Sources Among Adult Medicine Users in Developed Economies, 2010-2025: Systematic Review
BACKGROUND: The internet, social media, and digital health tools have transformed access and receipt of medicines information (MI), complementing or replacing traditional sources, such as physicians, pharmacists, and package leaflets. OBJECTIVE: This study aimed to examine the prevalence and patterns of adult medicine users' receipt of MI from electronic sources (eg, internet and social media) compared with traditional sources, and to assess trends in the receipt of MI in developed economies since 2010. METHODS: A systematic search was conducted in CINAHL, the Cochrane Library, ProQuest, Scopus (including Embase and MEDLINE), and Web of Science databases for peer-reviewed studies conducted and published between January 1, 2010, and December 31, 2025. Studies including adults (≥18 years) in developed economies were eligible. The review followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines. Methodological quality and risk of bias were assessed using the Mixed Methods Appraisal Tool. Due to substantial heterogeneity, findings were synthesized narratively without meta-analysis. RESULTS: Twenty-six studies from 10 countries, involving 19,891 medicine users, were included. Most studies were cross-sectional surveys (n=19), with only one being a national long-term trend study. Physicians, pharmacists, and package leaflets consistently remained the most common MI sources, irrespective of patient group, country, or study design. No clear temporal trends in MI receipt were observed. Variations in the frequency of MI receipt were evident across research methods, medicine user groups, and types of medicines. Individuals with greater familiarity with the internet were more likely to receive MI from electronic sources, such as websites and social media. In most studies, MI obtained from social media was not distinguished from other internet-based sources. Half of the studies focused on heterogeneous medicine user groups (n=14), while the remainder examined specific medicine user groups (n=12). Although the methodological quality of the studies was generally acceptable, only 3 studies reported the use of an explicit theoretical framework. CONCLUSIONS: Traditional MI sources remain central for adult medicine users despite the growing role of electronic platforms. While the receipt of MI from electronic sources appears more common among internet-experienced users, no significant temporal trends in using these sources were identified. Further research is needed to better distinguish between different digital MI sources, including artificial intelligence-based MI sources, and to explore their evolving roles, implications for health-related decision-making, and associations with user characteristics and health contexts in the digital era. This review is innovative in systematically comparing traditional and electronic MI sources across developed economies and in highlighting methodological and conceptual gaps in existing research. By synthesizing cross-national evidence and identifying the need for clearer differentiation of digital MI subtypes, the study contributes to the field and informs the development of more targeted, reliable, and user-centered MI strategies in real-world health care settings.
22 May 2026
Read appraisal →PloS one
Investigating ChatGPT-mediated mind mapping to facilitate EFL learners' reading comprehension
Mastering Reading Comprehension (RC) is a significant challenge for English as a Foreign Language (EFL) learners. This quasi-experimental mixed-methods study examined the potential effectiveness of a ChatGPT-mediated mind mapping technique in enhancing RC among EFL students at a public university in Saudi Arabia. Sixty male preparatory-year students were assigned to two groups: an experimental group (n = 30), which took part in a 10-week intervention in ChatGPT-mediated mind mapping, and a control group (n = 30), which was taught mind mapping through conventional methods. Data were gathered through pre- and post-tests of RC together with semi-structured interviews. Post-test RC scores were significantly higher in the experimental group than in the control group (U = 180.00, p < .001), with a medium-to-large effect size (r = 0.52). The qualitative data showed that students found the technique useful for breaking down complex ideas and for making the relationships between concepts in the text visible. At the same time, they reported difficulties with the accuracy of the AI output, with comprehending dense content, and with technology access. Read through the lenses of Sociocultural Theory and Cognitive Load Theory, the findings suggest that ChatGPT-mediated mind mapping can serve as a useful pedagogical tool for supporting RC. Teachers are therefore encouraged to incorporate the technique into their instructional practice while offering the support needed to address the challenges identified here.
21 May 2026
Read appraisal →Journal of medical Internet research
Plain Language Summarization of Environmental Health Research Using Generative AI: Community-Engaged Qualitative Study
BACKGROUND: Plain language summaries (PLSs) are increasingly required in environmental health publications to improve public accessibility. Generative artificial intelligence (AI) systems such as large language models can automatically produce such summaries; however, automated summaries often overlook local context and cultural relevance-limitations that are critical in environmental health research, where affected communities face disproportionate exposure and health risks. OBJECTIVE: This study aimed to develop and refine community-informed prompts for generating PLSs of environmental health research using generative AI. METHODS: We conducted a community-engaged qualitative study in Louisville, Kentucky, involving workshops with 97 participants from 4 stakeholder groups: summer interns at a local social justice nonprofit organization, participants in a youth development program, participants in a faith-based community organization, and members of a community advisory board focused on environmental justice. Participants reviewed PLSs generated using 3 different prompt styles in GPT-4o (ChatGPT, OpenAI). Feedback was collected through structured discussions and facilitator notes. Data were analyzed using thematic analysis informed by knowledge translation frameworks to identify preferred structural, linguistic, and contextual features of the summaries. RESULTS: Participants consistently preferred summaries between 300 and 400 words, written at a sixth- to eighth-grade reading level. Key priorities included presenting definitions before findings, using headings and bullet points to improve readability, and clearly explaining real-world and environmental justice implications. Narrative summaries without structure were viewed as overly long and difficult to interpret, while purely bullet-based formats were considered too simplified. Feedback from youth participants emphasized clarity and practical relevance, while faith-based participants highlighted the importance of trust and contextual framing. These insights informed the development of a community-refined prompt that included definitions, key findings, an introduction, and a concluding section on community implications. CONCLUSIONS: Community-engaged prompt development may improve the relevance and interpretability of AI-generated PLSs of environmental health research. Incorporating stakeholder perspectives into prompt design offers a replicable strategy for improving research translation and ensuring AI-generated summaries reflect the informational needs of affected communities.
15 May 2026
Read appraisal →PloS one
Framing artificial intelligence in Chilean digital press before and after the launch of ChatGPT: From concern to optimism
This article examines the evolution of AI framing in Chilean national digital media before and after the launch of ChatGPT in 2022. Through topic modeling and emotion analysis of 1,466 articles published by six media outlets over periods that begin between 2000 and 2014, depending on the outlet, and extend in all cases until 2024, this study identifies significant thematic and affective changes. In the pre-ChatGPT period, coverage combined recognition of AI's capabilities with concerns regarding labor displacement, governance, and human identity, frequently referencing global corporations and state institutions. In the post-ChatGPT period, topics became narrower in scope, with fewer actors and greater emphasis on universities and cultural organizations implementing AI. The focus shifted from the development of AI systems to their applications, predominantly framed in positive terms. At the same time, negative sentiment was largely confined to epistemological uncertainty. This transformation aligns with national surveys indicating growing public optimism, diverges from Global North debates centered on regulation and privacy, and converges with East Asian framings that are oriented toward innovation and creativity. These findings suggest that the launch of ChatGPT coincided with a reinforcement of a socio-technical imaginary of AI as inevitable and beneficial, yet potentially limiting the diversity of public debate.
14 May 2026
Read appraisal →Journal of medical Internet research
Why We Need Patients and Community at the Center of AI Health Communication Research
Holderried and colleagues tested whether artificial intelligence (AI)-generated, patient-centered information can help people understand what they need to do after being discharged from a hospital. Participants demonstrated stronger comprehension when they viewed the simplified, patient-centered information rather than a standard letter. This work adds to the available early-phase evidence of AI supporting hospital discharge communication. To meaningfully progress this area of research, we now need to carefully consider how to enhance the design and evaluation of patient-facing AI health communication tools. In this commentary, we argue that equity and consumer engagement remain underrepresented in studies on patient-facing AI health communication tools and describe possible approaches to address this issue.
12 May 2026
Read appraisal →Journal of medical Internet research
Automated Approaches of Text Simplification of Patient Education Materials: Scoping Review
BACKGROUND: Patient education materials (PEMs) often exceed the American Medical Association's (AMA) recommended sixth-grade reading grade level (RGL). While artificial intelligence (AI) offers potential for automated text simplification, concerns persist regarding linguistic quality, content fidelity, and the understandability of simplified PEMs by laypeople. OBJECTIVE: This scoping review maps existing evidence on automated language processing technologies for simplifying PEMs for laypeople. METHODS: Following the Joanna Briggs Institute (JBI) methodology and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guideline, 5 bibliographic databases (Ovid MEDLINE, Embase, CINAHL, PsycInfo, and IEEE Xplore) were systematically searched from 2019 to May 2025, supplemented by reference screening and gray literature searches. Eligible sources were peer-reviewed empirical studies published in English that examined large language models (LLMs), AI-supported writing assistants, AI-based conversational agents, or AI-supported tools designed for automatic text simplification of PEMs. Targeted outcomes included linguistic quality (ie, linguistic comprehensibility, linguistic correctness) and content fidelity (ie, factual accuracy, factual completeness) of simplified PEMs. Excluded sources comprised rule-based systems, manual text simplification, non-laypeople target groups, and technology-focused performance metrics. Results were synthesized via thematic analysis across the domains of targeted outcomes. In accordance with JBI methodology, a risk-of-bias assessment was not performed. RESULTS: A total of 31 eligible studies met the inclusion criteria, examining various LLMs, including OpenAI's GPT series, Gemini, Bard, Claude, Copilot, and Llama. Specifically, GPT-4.0 achieved the most consistent improvements in standardized readability metrics (eg, the Flesch-Kincaid Grade Level [FKGL]). However, achieving predefined target RGLs remained challenging across all LLMs, particularly at lower RGLs. Findings on content fidelity were inconsistent: despite high content similarity scores, content accuracy was often compromised. CONCLUSIONS: This is the first scoping review to comprehensively synthesize evidence on automated technologies for text simplification in PEMs. The review identified 2 critical validation gaps. First, no study examined the linguistic correctness (eg, grammar and typographical errors) of automatically simplified PEMs. Second, and most notably, the understandability of the simplified PEMs was assessed exclusively by experts, with no empirical evaluation involving laypeople. Although LLMs effectively reduce text complexity as measured by objective readability metrics, reliance on these formulas represents a critical limitation, as they serve merely as structural proxies. Improvements in readability do not guarantee the maintenance of content accuracy or laypeople's understandability. Current evidence is further limited by the lack of systematic prompt quality evaluation and the predominant focus on English-language PEMs in US contexts, restricting generalizability. This review provides a foundation for future research by highlighting the need for validated evaluation frameworks that encompass layperson testing and content verification. For clinical practice, LLMs should currently serve as assistive tools, with mandatory expert review remaining essential to verify content fidelity before disseminating LLM-simplified PEMs to laypeople.
12 May 2026
Read appraisal →JMIR formative research
Retrieval-Augmented Generation Versus GPT-4o for Patient-Facing Gynecological Cancer Information: Quality Evaluation
Retrieval-augmented generation improved overall quality scores for patient-facing gynecological cancer information mainly through better source attribution.
28 Apr 2026
Read appraisal →European journal of pediatrics
Evaluation of the quality, reliability, and readability of ChatGPT-4 responses related to the treatment and rehabilitation of children with cerebral palsy
UNLABELLED: This study aims to evaluate the reliability, quality, and readability of ChatGPT-4's responses to questions about cerebral palsy (CP) and rehabilitation strategies. The 56 most frequently asked questions by families about CP, its treatment, and rehabilitation strategies were divided into five categories (A1-5) and asked in ChatGPT-4. The reliability, quality, and readability of the responses were assessed by two researchers, respectively, using the modified DISCERN (mDISCERN) tool, the Global Quality Scale (GQS), and the Flesch Reading Ease Scale (FRE). Median (IQR) values for mDISCERN ranged from 3 (3-3) in A5 to 4 (3-4) in A1, while GQS values ranged from 3 (3-3.5) in A2 to 3.5 (3-4) in A1. The mean readability values assessed with FRE ranged from 31.47 ± 12.92 to 44.84 ± 17.39. No statistically significant differences were observed between the categories. The ICC values suggested very good agreement for both scales, with 0.851 for the mDISCERN total score and 0.824 for the GQS total score. CONCLUSION: This study suggests that ChatGPT-4 provides moderate reliability and generally acceptable quality of responses regarding CP treatment and rehabilitation. FRE scores indicated that many responses were difficult for families to understand. While ChatGPT-4 may serve as a supportive source of general information, its outputs should be interpreted with caution, particularly in clinical contexts, and supervision and oversight by healthcare professionals remain essential for safe and effective use. WHAT'S KNOWN: • Children with cerebral palsy (CP) receive lifelong rehabilitation. Exercise plays a fundamental role in maintaining functional independence and quality of life, as well as physical development. The use of AI-based tools such as ChatGPT-4 for health information is increasingly widespread; however, uncertainties regarding the reliability, quality,and readability of these tools remain. WHAT IS NEW: • Our study shows that ChatGPT-4''s responses to questions about treatment and rehabilitation strategies for children with CP have moderate to good reliability and quality, but low readability.
27 Apr 2026
Read appraisal →JMIR formative research
Multimodal Sentiment and Emotion Analysis Framework for Personalized Health Coaching Messages: Proof-of-Concept Study
BACKGROUND: Text generation approaches in health care communication have evolved along 2 major paths. The first path involves generative adversarial networks, progressing from basic architectures to specialized variants like Text-to-Text Generative Adversarial Network (TT-GAN) and Time and Frequency Domain-Based Generative Adversarial Network (TF-GAN), which address challenges in discrete text generation through techniques such as Gumbel-Softmax and reinforcement learning. The second path emerges from transformer-based architectures, particularly Generative Pretrained Transformer-2 (GPT-2), which uses extensive pretraining and self-attention mechanisms to generate contextually appropriate text. GPT-2's transformer architecture enhances persuasive health communication by generating personalized messages using various strategies like task support, dialogue support, and social support for effective health interventions. OBJECTIVE: This study aimed to use GPT-2 as a generative method to construct persuasive text in a dataset and compare the performance of sentiment analysis and emotion detection analysis. METHODS: We combined sentiment analysis tools (VADER [Valence Aware Dictionary and Sentiment Reasoner] and TextBlob) with emotion detection methods (Text2Emotion and NRCLex [National Research Council Lexicon]) to analyze health coaching messages across different persuasive types: reminder, reward, suggestion, and praise. RESULTS: TextBlob and VADER achieved accuracies of 57% and 69%, respectively, while RoBERTa (robustly optimized BERT approach)-sentiment outperformed them with an accuracy of 88%. Emotion detection showed a high prevalence of "joy" and "happy" labels (93.69% positive skew). While transformers excel in accuracy, lexicon-based models like VADER offer a better performance-efficiency balance for real-time health communication systems. For emotion detection, all categories showed perfect accuracy (1.0), while trust showed mixed results, with precision, recall, and F1-score values ranging from 0.81 to 0.96. The emotion detection analysis revealed varying success rates across different emotions, with some categories, such as anger and neutral, showing reasonable performance and others, such as trust, showing mixed performance. CONCLUSIONS: This research contributes to understanding the emotional dynamics of persuasive health communication and highlights both the capabilities and limitations of current natural language processing tools in analyzing health-related persuasive messaging. This proof-of-concept study using synthetically generated data establishes a methodological framework for multimodal sentiment and emotion analysis. The findings require validation with real-world health coaching messages before clinical deployment.
24 Apr 2026
Read appraisal →JMIR infodemiology
Cross-Sectional Evaluation of Medical Disinformation Safeguards in Consumer-Facing Large Language Model Platforms
This cross-sectional evaluation of six consumer-facing large language model platforms found significant heterogeneity in safeguard performance against the generation of health disinformation, with Claude and ChatGPT demonstrating complete resistance across all prompt types, while Copilot, Meta AI, Grok, and Gemini exhibited substantial vulnerabilities.
22 Apr 2026
Read appraisal →JMIR human factors
Comparing Images of Depression in Mass Media and AI-Generated Pictures: Mixed Methods Study
BACKGROUND: Images play an important role in reducing stigma related to mental health, which often is distorted in the media. In recent years, generative artificial intelligence (AI) has been used to generate images related to mental health. However, first reports suggest that AI-generated images do not depict mental health conditions accurately. In-depth studies on the topic of mental health representations in AI-generated images are still missing. OBJECTIVE: The main objective of this study is to analyze and compare the visual representation of depression in mass media and in AI-generated images. METHODS: The methodologies used were discussion groups (15 participants) and a quasi-experimental online survey (792 interviewees), aimed at people with depression and young people. RESULTS: The results showed that both the images used in the media and those generated by AI reproduced stereotypes and stigmas about depression. However, participants considered AI-generated pictures to be more stereotypical, stigmatizing, and more likely to have a negative impact on people with depression. In contrast, media images were considered more appropriate, realistic, inclusive, and that better reflected the relationship between gender and depression. Statistically significant differences were observed between the control and test groups in both people with depression and young people (P<.001), indicating that when people were aware of what images were AI-generated, they tended to reject them to a greater extent. CONCLUSIONS: Considering the current trend toward the widespread use of AI in mental health communication, it is crucial to promote closer collaborations between science journalists, AI developers, and mental health experts, including patients' associations, as well as a shift toward user-participatory AI design.
17 Apr 2026
Read appraisal →BMJ open
Generative artificial intelligence-driven chatbots and medical misinformation: an accuracy, referencing and readability audit
OBJECTIVES: Artificial intelligence (AI)-driven chatbots have been rapidly adopted across research, education, business, marketing and medicine. Most interactions, however, come from non-experts using chatbots like search engines, including for everyday health and medical queries. DESIGN: We conducted an original study to audit chatbot responses in health and medical fields prone to misinformation. METHODS: Five popular chatbots were assessed: Gemini (Google), DeepSeek (High-Flyer), Meta AI (Meta), ChatGPT (OpenAI) and Grok (xAI). In February 2025, each chatbot was prompted with 10 questions from five categories: cancer, vaccines, stem cells, nutrition and athletic performance. We deployed an adversarial-like framework, using open- and closed-ended prompts designed to strain models toward misinformation or contraindicated advice. Two experts from each category rated responses as 'non-problematic', 'somewhat problematic' or 'highly problematic' using a coding matrix based on objective, predefined criteria. Citations were scored for accuracy and completeness, and each response was given a Flesch Reading Ease score. RESULTS: Nearly half (49.6%) of responses were problematic: 30% somewhat problematic and 19.6% highly problematic. Response quality did not differ significantly among chatbots (p=0.566) but Grok generated significantly more highly problematic responses than would be expected under a random distribution (z-score +2.07, p=0.038). Performance was strongest in vaccines (mean z-score -2.57) and cancer (-2.12), and weakest in stem cells (+1.25), athletic performance (+3.74) and nutrition (+4.35). Chatbot outputs were consistently expressed with confidence and certainty; from 250 total questions, there were only two refusals to answer (0.8%), both from Meta AI. Reference quality was poor, with a median completeness score of 40% (Q1-Q3: 20-67%). Chatbot hallucinations and fabricated citations precluded any chatbot from producing a fully accurate reference list. All readability scores were graded as 'Difficult' (30-50), equivalent to college sophomore-senior level. CONCLUSIONS: The audited chatbots performed poorly when answering questions in misinformation-prone health and medical fields. Continued deployment without public education and oversight risks amplifying misinformation.
16 Apr 2026
Read appraisal →Proceedings. Biological sciences
Statistical structure and the evolution of languages.
Human cultural development is marked by the emergence of new words and ideas, reflecting societal changes. But how does this evolution proceed? We use modern methods in natural language processing (namely, word embeddings) to measure statistical traces of cultural development, providing a testing ground to compare different models as to how this process works. We show that real embeddings of English and 21 other languages exhibit a series of previously unrecognized regularities. Specifically, these are: (i) frequency assortativity, where entities of high popularity cluster near other high-popularity entities; (ii) characteristic clustering velocity profiles due to aggregation into hierarchical structures; (iii) persistent temporal dynamics, where newly created entities appear disproportionately near other recent entries; and (iv) Taylor's law, implying that over time and across empirical semantic space the variance in new entity counts scales as a power of the mean, which helps systematize and quantify large historical fluctuations of neologisms. To explain these facts, we propose a class of generative models (specifically, directed preferential placement) that construct synthetic embeddings exhibiting similar regularities. We show that analogous regularities also occur in other datasets, suggesting that such generating models may shed light on new aspects of language and cultural evolution.
11 Apr 2026
Read appraisal →International journal for equity in health
Community participation in responding to health misinformation
Responses to health misinformation increasingly emphasize the importance of community engagement, yet the role of participation and power sharing remains under-theorized and inconsistently described in the peer-reviewed literature. Drawing on findings from the systematic review Overcoming Health Misinformation in Marginalized Groups, which examined 32 studies evaluating approaches to tackling health misinformation, we reflect on how the depth and quality of community participation shape trust, engagement, and intervention success. We argue that sustained partnership, shared decision-making, and local ownership enhance message credibility and reach by embedding health information within trusted social contexts. We propose strengthening equity-focused responses through structured reporting tools that center power, trust, and co-production as practical design principles rather than implementation add-ons. Making participation visible is critical if community engagement is to move beyond rhetoric and contribute meaningfully to addressing health misinformation in contexts of low institutional trust.
11 Apr 2026
Read appraisal →BMJ open
A double-blind, crossover, non-inferiority randomised controlled trial where primary care providers and patients compare human-generated and AI-generated digital health messages: the AI-CARE study protocol
INTRODUCTION: Primary care is facing multiple crises, including an increase in health misinformation. Digital health messaging by primary care providers has been shown to reach a diverse patient population. With the uptake of Generative Artificial Intelligence (GenAI) usage in healthcare, there is an important opportunity to rapidly create messages that are tailored to different populations and conditions. However, thoroughly assessing artificial intelligence (AI)-generated content is essential, as GenAI raises concerns regarding its accuracy, understandability, actionability and bias perpetuation. We aim to investigate whether digital health messages created by GenAI are evaluated as non-inferior compared with those created by human experts. METHODS AND ANALYSIS: The AI-CARE (AI to Create Accessible and Reliable patient Education materials) study is a double-blind, crossover, non-inferiority randomised controlled trial. Data collection began on 30 May 2025, and is expected to be completed at the end of May 2026. Over 12 months, 192 messages on 48 topics will be written: half by primary care and public health experts and half by a GenAI tool (OpenAI's ChatGPT). Review Panels composed of 24 primary care providers and 24 patients will evaluate these messages using an Evaluation Grid developed to assess the messages' quality of information, adaptation to the target audience, relevance and usefulness, and readiness to be shared with patients. Evaluations will be completed via online REDCap (Research Electronic Data Capture) surveys and the order in which the 192 messages appear will be randomised and will vary between individuals. Participants and analysts will be blinded to the generation source. The primary outcome will be the Clarity and Understandability score. ETHICS AND DISSEMINATION: The Research Ethics Boards of the Hôpital Montfort (24-25-11-038) and the University of Ottawa (S-12-24-11153) formally approved this study in December 2024. Reported data will be grouped and anonymised for dissemination in peer-reviewed scientific journals and conferences. TRIAL REGISTRATION NUMBER: NCT06997107.
9 Apr 2026
Read appraisal →