Summary:
AI-generated summaries can manipulate eyewitness memory, causing people to misremember events they witnessed firsthand. In a controlled experiment, exposure to misleading automated summaries nearly halved participants’ recall accuracy, implanting false memories even when viewers were explicitly informed the summaries came from AI.
Key Facts:
- Severe Omission Rates: Across consumer multimodal AI models (including ChatGPT and Gemini), automated video summaries omitted an average of 51.6% of central events, with 95% failing to mention the incident’s core occurrence: a car hitting a pedestrian.
- Recall Accuracy Halved: Participants who read an accurate summary correctly recalled key scene details 83.6% of the time, compared to only 44.8% of those exposed to an inaccurate AI summary.
- The “Human-in-the-Loop” Fallacy: Labeling a summary as AI-produced failed to insulate observers from memory contamination; participants internalized errors regardless of their stated familiarity with or trust in artificial intelligence.
Source: Georgetown University / University of Washington
From corporate meeting transcripts and clinical case notes to law enforcement body-worn camera logs, institutions are accelerating the deployment of large language models to condense long video and audio streams into concise narrative digests. Yet cognitive psychologists have long cautioned that human episodic memory is not a fixed video recording, but a malleable reconstructive process vulnerable to post-event misinformation.
A joint study conducted by researchers at Georgetown University and the University of Washington reveals that generative AI tools can serve as potent vectors for memory distortion.
Presented at the Ninth AAAI/ACM Conference on AI, Ethics, and Society (AIES), the research demonstrated that misleading AI-generated video summaries systematically alter an eyewitness’s recollection of past events—even when the reader knows the text was written by a machine.
“AI is a new method of delivering misinformation, and it has the potential to create these false memories for people who are reading that information,” said lead author Mattea Sim, an assistant research professor at the Massive Data Institute in Georgetown’s McCourt School of Public Policy. “I think we should be thinking really deeply and critically about whether and how AI should be used to summarize information, especially in high-stakes settings.”
Missing the Collision: Massive Omission in Video Condensation
The researchers investigated two primary dimensions of AI integration: the baseline factual fidelity of commercial models summarizing footage, and the downstream cognitive consequences for human observers.
First, the investigators tested OpenAI’s ChatGPT and Google’s Gemini on animated traffic incident videos adapted from landmark psychological paradigms on eyewitness reliability. The AI outputs exhibited substantial descriptive deficits, recurrent hallucinations, and systemic omissions.
On average, the models failed to mention 51.6% of central scene events. In 95% of tested iterations, the tools omitted the most consequential detail depicted in the footage: a motor vehicle colliding with a crossing pedestrian.
“I was struck by how bad the summaries were, even at this stage in AI development,” said study co-author Yael Eiger, a Ph.D. candidate at the University of Washington. “It worries me that police departments may be using video summarization technologies without rigorous testing and without an awareness of how incorrect AI-generated summaries could be.”
Implanting Misinformation Across 48 Hours
To quantify the cognitive fallout, the research team recruited 331 participants who viewed animated videos of a red car approaching an intersection governed by either a stop sign or a yield sign before turning and hitting a pedestrian.
Between 24 and 48 hours later, participants reviewed a narrative summary of the incident. Some cohorts received factually precise accounts, while others read texts containing altered, erroneous details. Additionally, researchers altered the perceived provenance of the text, informing subjects that the digest was prepared either by an automated AI model or a human transcriber.
When tested on their recollection of the original event, participants exposed to misleading texts demonstrated severe memory impairment:
- 83.6% Accuracy among viewers who read an accurate summary.
- 44.8% Accuracy among viewers exposed to an inaccurate summary.
Crucially, knowing an AI wrote the text offered zero cognitive defense. The rate of false memory acceptance remained uniform whether participants believed the text originated from a machine or a human clerk. Individual baseline trust and prior familiarity with artificial intelligence similarly failed to buffer against memory contamination.
Questioning the “Human-in-the-Loop” Safety Valve
These findings directly challenge the common administrative premise that placing a “human-in-the-loop” provides a reliable safeguard against AI hallucinations.
“Though ‘humans-in-the-loop’ are often expected to correct for AI’s mistakes, our work suggests human memory can instead be distorted by these mistakes,” the authors emphasized. “AI has the potential to generate misinformation, even absent any adversarial intent, which can meaningfully impact human memory.”
In judicial and policing contexts, an officer or eyewitness who reviews an erroneous AI-generated incident summary prior to writing a deposition may inadvertently adopt those automated errors as authentic memories.
“This study is part of a growing effort at Georgetown focused on exploring the impact and relationship between AIs and humans, grounded in both psychology and computer science,” noted Yoshi Kohno, McDevitt Chair in Computer Science, Ethics, and Society at Georgetown University and co-author of the work.
Moving forward, the researchers plan to transition from synthetic animations to real-world police body-worn camera footage, aiming to map how commercial summarization tools alter official reports and civilian testimony in legal proceedings.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- Journal paper reviewed in full.
- Additional context added by our staff.
About this AI and memory Research:
- Media Contact: Jason Shevrin
- Source: Georgetown University
- Image Credit: Image credited to Neuroscience News
- Original Research is Open Access: arXiv (Sept 23, 2026). “AI-Enabled Human Memory Manipulation: Misleading AI-Generated Summaries Distort Human Memory.” Authors: Mattea Sim, Yael Eiger, and Tadayoshi Kohno.
- DOI: 10.48550/arXiv.2609.28820
Abstract
AI-Enabled Human Memory Manipulation: Misleading AI-Generated Summaries Distort Human Memory
AI-generated summaries are increasingly used in high-stakes settings, like policing, despite considerable evidence that AI often generates misleading or inaccurate information. This research asked: do errors in AI-generated summaries distort human memory?
To answer this question, we adopted two methodological approaches.
First, we conducted an analysis of AI summary output, prompting large language models to generate summaries of videos. This analysis quantified how often AI summaries contain errors and the categories of these errors, revealing the kinds of misleading information that may distort human memory.
Second, we conducted a human-subjects experiment to test the impact of misleading information in AI-generated summaries on human memory. Participants were first exposed to an event via watching a video of a car-pedestrian accident, and later read an AI-generated summary describing the video that either contained misleading or accurate information.
Participants’ memory for the original event was assessed in a memory recognition test. In the AI analysis, we found a high frequency of mistakes in AI summaries, and particularly frequent omissions of critical details. For instance, the majority of summaries omitted the most central event of the video, a critical error which is likely to be impactful.
We also observed a strong effect of AI misinformation on human memory. People who read a misleading AI summary were significantly less likely to accurately recall the original event, compared to people who read an accurate AI summary.
These findings have implications for how AI should be used in critical settings. Though “humans-in-the-loop” are often expected to correct for AI’s mistakes, our work suggests human memory can instead be distorted by these mistakes. AI has the potential to generate misinformation, even absent any adversarial intent, which can meaningfully impact human memory.
