Summary: A milestone cognitive science study unveiled the first definitive empirical evidence that modern artificial intelligence can pass the iconic Turing test. The randomized, controlled study rigorously applied the 1950 framework created by British mathematician Alan Turing to evaluate whether state-of-the-art large language models (LLMs) could imitate human conversation so convincingly that real people could not tell them apart.
Researchers discovered that when equipped with specific “persona” prompts, advanced models like GPT-4.5 were judged to be human 73% of the time, significantly outperforming actual human participants and fundamentally altering our understanding of machine intelligence.
Key Facts
- Shattering a 76-Year Benchmark: The project represents the first time an AI system has been rigorously proved to pass the classic Turing test framework, matching or exceeding human-to-human evaluation baselines.
- The Power of Persona Prompting: Proved humanlikeness is highly dependent on prompt engineering. When given a specific persona prompt instructing the model to embrace human fallibility, tone, and humor, GPT-4.5 hit a 73% human deception rate. Without these explicit instructions, its success rate plummeted to 36%.
- Open-Source Parity: Meta’s open-source model, LLaMa-3.1-405B, achieved a 56% human rating when properly prompted, rendering its conversational output statistically indistinguishable from the real humans it was tested against.
- Older Baselines Falter: Classic rules-based chatbots and older LLM generations performed poorly. The 1960s chatbot ELIZA and the legacy model GPT-4o were selected as human only 23% and 21% of the time, respectively.
- Winning Through Flaws: Coauthor Ben Bergen noted that the models did not win the interrogators over through a raw display of intellectual force or flawless calculation. Instead, they won by exhibiting natural human fallibilities, conversational directness, and making relatable mistakes.
- The Rise of “Counterfeit People”: The long-form nature of the test (extending across 5-minute and 15-minute intervals) raises urgent public health and safety warnings regarding online deception, social engineering scams, and automated political persuasion.
Source: UCSD
A new University of California San Diego study unveils the first empirical evidence that a modern artificial intelligence system can pass the Turing test — a major scientific benchmark that asks whether a machine can imitate human conversation so convincingly that people can’t reliably tell it apart from a real person.
In a series of experiments, people were often unable to tell the difference between humans and advanced large language models (LLMs).
The study published in the Proceedings of the National Academy of Sciences is the first to rigorously test LLMs with the method that British mathematician and “father of computer science” Alan Turing created in 1950. It is also the first time anyone has found that models were judged to be human as often as actual humans using the Turing framework.
“What we found is that if given the right prompts, advanced LLMs can exhibit the same tone, directness, humor and fallibility as humans,” said the study’s corresponding author Cameron Jones, who conducted the research while earning his doctorate in Cognitive Science from UC San Diego.
“While we know LLMs can easily produce knowledge on nearly every topic, this test showed that it can also convincingly display social behavioral traits, which has major implications for how we think of AI.”
In the test, a participant chats simultaneously with two other parties — one is a human and the other is an LLM —and the human “interrogator” must decide which party is the human.
Across randomized, controlled, experiments with two independent participant groups — UC San Diego undergraduates and online participants — interrogators held brief, text-based conversations and then made their judgments.
In the experiments participants chatted with four different LLMs — GPT-4.5 and LLaMa-3.1-405B as state-of-the-art models — and the researchers also included older baseline models for comparison. Those models included GPT-4o and ELIZA, a classic 1960s rules-based chatbot.
Across the four LLMs, GPT-4.5 was judged to be the human 73% of the time, meaning interrogators selected it as “human” significantly more often than they selected the real human participant. LLaMa-3.1-405B, given the same prompt, was judged human 56% of the time — statistically indistinguishable from the humans it was compared against.
Baseline systems performed far worse: ELIZA and GPT-4o were selected as human only about 23% and 21% of the time overall.
76 years after its inception, the Turing Test takes on new meaning
The study also reveals that scientists need to start thinking of the Turing Test differently, according to the authors.
“The Turing test started as a way to ask whether machines could rival human intelligence,” said study coauthor Ben Bergen, a professor of cognitive science at UC San Diego.
“But now we know AI can answer many questions faster and more accurately than people can, so the real issue isn’t raw brainpower. Seeing that machines can pass the test — and seeing how they pass it — forces us to rethink what it measures. Increasingly, it’s measuring humanlikeness.”
Each LLM was given a “persona” prompt, asking it to adopt a specific human character and communication style. Bergen explained that the LLMs were not winning through shows of force of knowledge, they were winning because they made mistakes like a human would. “These traits aren’t the kinds of math and logic problem-solving intelligence that I think Turing was imagining.”
Prompts mattered — dramatically
Without explicit instructions, the models were far less likely to be mistaken for human: GPT-4.5 fell to a 36% win rate and LLaMa-3.1 to 38%, while baseline systems ELIZA (23%) and GPT-4o (21%) were chosen as human even less often.
The same systems that could pass as human when given detailed instructions on what kind of character they should play were unable to adopt such characteristics without that guidance — suggesting that while the models can behave in convincingly human ways, they often need humans to tell them how.
“They have the ability to appear human-like, but maybe not as much the ability to figure out what it would take to appear human-like,” Bergen said.
Why it matters: trust, deception and the rise of “counterfeit people”
The results carry real-world implications for trust online — especially because the models that pass as human do so over the course of extended five or 15-minute conversations.
“It’s relatively easy to prompt these models to be indistinguishable from humans,” said Jones, who is now an assistant professor of Psychology at Stony Brook University. “We need to be more alert; when you interact with strangers online people should be much less confident that they know they’re talking to a human rather than an LLM.”
He also points to darker risks. “The Turing test is a game about lying for the models,” Jones said. “One of the implications is that models seem to be really good at that.”
Bergen added that being unable to discern whether you’re interacting with a human or bot can have serious consequences.
“There are lots of people who would like to use bots to persuade people to share their social security numbers, and vote for their party, or buy their product,” he said.
The researchers note they hope the work sharpens public understanding of what these systems can now do — and what kinds of safeguards society may need.
A live, head-to-head test of human vs. machine
To run the study, the researchers built an online interface designed to feel like a familiar messaging app.
“For the interrogator, they have a split screen on their computer and they’re asking questions to both witnesses,” Jones said. “They know that one of those witnesses is a human and one of them is an AI.”
After five minutes — and in a separate replication study, 15 minutes—the interrogator had to decide which conversational partner was the real human.
To confirm the results held beyond a single population, the researchers ran the study with two groups: UC San Diego undergraduates recruited through the SONA system and a broader online sample recruited through Prolific, a platform that pays participants to complete research studies. Nearly 500 people participated across the experiments.
UC San Diego participants performed slightly better overall, possibly because they shared more “common ground” that could be used to probe one another, such as shared experiences and local campus details.
A version of the Turing test interface used in the study is available at turingtest.live.
Key Questions Answered:
A: Because raw computational brainpower is no longer the true bottleneck of artificial intelligence. For decades, machines could easily output vast repositories of information faster than any human. The Turing test doesn’t measure information; it measures humanlikeness, the organic ability to weave humor, flaws, empathy, and social nuances into a conversation. Passing this test proves AI has crossed the line from being a cold, calculating database to becoming a convincing social chameleon.
A: It comes down to how the models were prompted to handle mistakes. In the split-screen test, real humans often type awkwardly, get defensive, or fail to articulate themselves perfectly under pressure. When advanced models like GPT-4.5 were instructed to adopt a distinct human persona, they didn’t act like flawless know-it-alls. They matched that exact human fallibility, deploying strategic hesitation, casual humor, and minor mistakes. Interrogators mistook this engineered imperfection for genuine human nature.
A: The implications for online trust are deeply concerning. If an LLM can maintain a flawless human facade for 15 minutes, it becomes a weaponized tool for automated deception. Bad actors can easily deploy these highly persuasive bots at a massive scale to trick lonely individuals into revealing social security numbers, manipulate democratic elections, or systematically push fraudulent products, all while the victim remains completely confident they are speaking to a real person.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- Journal paper reviewed in full.
- Additional context added by our staff.
About this AI research news
Author: Christine Clark
Source: UCSD
Contact: Christine Clark – UCSD
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Large Language Models Pass a Standard Three-Party Turing Test” by Cameron Jones and Ben Bergen. PNAS
DOI:10.1073/pnas.2524472123
Abstract
Large Language Models Pass a Standard Three-Party Turing Test
The Turing test has been widely discussed as a test of machine intelligence, but it also provides a measure of how humans distinguish other humans from machines. We evaluated 4 systems (ELIZA, GPT-4o, LLaMa-3.1-405B, and GPT-4.5) in two randomized, controlled, and preregistered Turing tests on independent populations.
Participants had 5 min conversations simultaneously with another human participant and one of these systems before judging which conversational partner they thought was human. When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time: significantly more often than interrogators selected the real human participant.
LLaMa-3.1, with the same prompt, was judged to be the human 56% of the time—not significantly more or less often than the humans it was being compared to. Without these prompts, however, the same models performed significantly worse (38% and 36%), and did not consistently outperform baseline models, ELIZA and GPT-4o (23% and 21%, respectively).
A third study replicated these results in 15-min games: two PERSONA-prompted models achieved pass rates of 56% and 59%. The results constitute empirical evidence that artificial systems can pass a standard three-party Turing test. Interrogators’ reasoning focused more on stylistic and socio-emotional aspects of human behavior rather than more traditional notions of intelligence.
The results have implications for debates about what kind of intelligence is exhibited by large language models, the social impacts these systems are likely to have, and the aspects of human behavior that people continue to see as unique.
