Summary: A new study warns of “AIPasta,” a strategy that uses generative AI to craft many slightly different versions of false claims to mimic widespread public belief. Unlike traditional CopyPasta, which repeats the same message verbatim, AIPasta increased perceptions of consensus, especially among politically predisposed groups.
In experiments, AIPasta raised belief in election and pandemic conspiracies among Republicans more effectively than CopyPasta. Researchers caution that AIPasta evades detection and may be harder to moderate on social media platforms.
Key Facts:
- New Threat: AIPasta combines AI and CopyPasta to spread varied, persuasive falsehoods.
- Consensus Illusion: AIPasta made false claims appear more widely believed than CopyPasta.
- Harder to Detect: AIPasta text evaded AI detection tools, unlike traditional CopyPasta.
Source: PNAS Nexus
Brace yourself for a new source of online disinformation: AIPasta.
Research has demonstrated that generative AI can produce persuasive content. Meanwhile, so-called CopyPasta campaigns take advantage of the “repetitive truth” effect by repeating the exact same text over and over until it seems more likely to be true by those who encounter it many times.
Saloni Dash and colleagues explore how these two strategies can be combined into what the authors term “AIPasta.”
In AIPasta campaigns, AI can be used to produce many slightly different versions of the same message, giving the public the impression that the message is widely held by many different people and likely to be true.
The authors used both CopyPasta and AIPasta methods to produce messaging around the conspiracy theories that the 2020 presidential election was fraudulent or that the COVID-19 pandemic was intentional.
In an online survey of 1,200 Americans recruited via Prolific, neither CopyPasta nor AIPasta were effective in convincing study participants that the studied conspiracy theories were true.
When examining just Republican participants, who might be predisposed to give credence to the specific conspiracies studied, AIPasta did increase belief in the false claim of the campaign more than CopyPasta.
However, for participants of both parties, exposure to AIPasta—but not CopyPasta—increased the perception that there was broad consensus that the claim was true.
According to the authors, the AIPasta generated for the study was not detected by AI-text detectors, suggesting it will be harder to remove from social media platforms than CopyPasta, which is likely to amplify its effectiveness compared to CopyPasta.
About this AI and disinformation research news
Author: Saloni Dash
Source: PNAS Nexus
Contact: Saloni Dash – PNAS Nexus
Image: The image is credited to Neuroscience News
Original Research: Open access.
“The persuasive potential of AI-paraphrased information at scale” by Saloni Dash et al. PNAS Nexus
Abstract
The persuasive potential of AI-paraphrased information at scale
In this article, we study how AI-paraphrased messages have the potential to amplify the persuasive impact and scale of information campaigns.
Building from social and cognitive theories on repetition and information processing, we model how CopyPasta—a common repetition tactic leveraged by information campaigns—can be enhanced using large language models.
We first extract CopyPasta from two prominent disinformation campaigns in the United States and use ChatGPT to paraphrase the original message to generate AIPasta.
We then validate that AIPasta is lexically diverse in comparison to CopyPasta while retaining the semantics of the original message using natural language processing metrics.
In a preregistered experiment comparing the persuasive potential of CopyPasta and AIPasta (N = 1,200), we find that AIPasta (but not CopyPasta) is effective at increasing perceptions of consensus in the broad false narrative of the campaign while maintaining similar levels of sharing intent with respect to Control (CopyPasta reduces such intent).
Additionally, AIPasta (vs. Control) increases belief in the exact false claim of the campaign, depending on political orientation.
However, across most outcomes, we find little evidence of significant persuasive differences between AIPasta and CopyPasta. Nonetheless, current state-of-the-art AI-text detectors fail to detect AIPasta, opening the door for these operations to scale successfully.
As AI-enabled information operations become more prominent, we anticipate a shift from traditional CopyPasta to AIPasta, which presents significant challenges for detection and mitigation.
