Large language models pass a standard three-party Turing test
Clinical Snapshot
PICO Framework
| P — Population | Human participants engaging in conversational interactions |
| I — Intervention | Not applicable - this is a computer science/artificial intelligence study |
| C — Comparator | Not applicable - this is a computer science/artificial intelligence study |
| O — Outcomes | Not applicable - this is a computer science/artificial intelligence study |
Bottom Line
This paper reports on artificial intelligence research evaluating large language models' ability to pass Turing tests - a computer science benchmark for machine intelligence. The study found that certain AI systems could be mistaken for humans in conversational tasks when given appropriate prompts. While this represents an interesting technological development, it is not a clinical study and provides no evidence relevant to patient care, diagnostic accuracy, therapeutic interventions, or health outcomes. Healthcare professionals should note that this research does not inform clinical decision-making or evidence-based medicine practices.
Key Findings
Primary Outcome: Not applicable - no clinical outcomes
Clinical Application
Not applicable to clinical practice Not relevant to Australian healthcare practice Not applicable to clinical populations
Abstract
The Turing test has been widely discussed as a test of machine intelligence, but it also provides a measure of how humans distinguish other humans from machines. We evaluated 4 systems (ELIZA, GPT-4o, LLaMa-3.1-405B, and GPT-4.5) in two randomized, controlled, and preregistered Turing tests on independent populations. Participants had 5 min conversations simultaneously with another human participant and one of these systems before judging which conversational partner they thought was human. When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time: significantly more often than interrogators selected the real human participant. LLaMa-3.1, with the same prompt, was judged to be the human 56% of the time-not significantly more or less often than the humans it was being compared to. Without these prompts, however, the same models performed significantly worse (38% and 36%), and did not consistently outperform baseline models, ELIZA and GPT-4o (23% and 21%, respectively). A third study replicated these results in 15-min games: two PERSONA-prompted models achieved pass rates of 56% and 59%. The results constitute empirical evidence that artificial systems can pass a standard three-party Turing test. Interrogators' reasoning focused more on stylistic and socio-emotional aspects of human behavior rather than more traditional notions of intelligence. The results have implications for debates about what kind of intelligence is exhibited by large language models, the social impacts these systems are likely to have, and the aspects of human behavior that people continue to see as unique.
References
- 1.Jones, C. R., & Bergen, B. K. (2026). Large language models pass a standard three-party Turing test. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2524472123
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service