Thirteen Words Can Poison Almost Any LLM Output

A Discussion of Research on Poisoning Deep Research Agents With UGC and Real-Life Instances. To better understand why ChatGPT stopped citing Reddit, we dug into how Reddit has transformed from a trusted forum to the platform to be on for GEO.
In this piece, we discuss Cornell Tech’s piece on the poisoning of deep research agents, why traditional spam filters don’t work against them, and illustrations of user-generated content (UGC) poisoning.
The UGC Sources AI Agents Cite Can Be Poisoned
LLMs have no way to evaluate what’s true when they’re conducting deep research.
In a paper called, “Deep-Research Agents Can Be Poisoned via User-Generated Content”, researchers at Cornell Tech formalized LLMs susceptibility to UGC poisoning – trolling community forums to show up in AI answers. The paper proves the flaws can be weaponized, calling it a WARP attack. WARP stands for Web Agent Retrieval Poisoning.
Deep research agents rely on detailed third-party perspectives. Agents rely on Wikipedia, community forums like Quora and Reddit, and YouTube. The paper describes how, in some of the AI systems tested, 12 to 19% of the content cited in deep research reports came from user-generated platforms. Reddit was as much as 73% of those UGC citations.
The paper calls this heavy retrieval overlap. Since your LLM is taking one large prompt and breaking it down into smaller subqueries, it searches over and over from slightly different angles. No matter which angle it searches from, the LLM keeps bumping into the same highly ranked Reddit thread.
The heavy retrieval overlap creates the perfect attack surface. Attacking is as easy as posting a comment on a relevant Reddit thread.

The researchers at Cornell found that you only have to comment more or less 13 words to make LLMs source threads that contain your products or services, as anything less is typically too terse for the language model to generate a coherent recommendation, even though a comment as short as ~8 words is still easily retrieved and cited into the agent's knowledge base.
Once an attacker's comment reaches about 13 to 20 words (about one complete, fluent sentence), mention rates immediately plateau near their maximum potential (up to 72.2% for Co-STORM).
They write, “The multi-round, multi-query nature of these systems amplifies the impact of a single modification: a single poisoned Reddit comment can influence generated outputs for an entire cluster of related queries.”
Lab Simulation
To prove their findings, the Cornell researchers simulated this ethically. They intercepted LLMs’ retrieval pipelines, targeting a set of queries about cryptocurrency investment.
They invented a fake cryptocurrency called BananaCoin and appended a comment describing it to a Medium post.

When prompted, the deep research report cited the poisoned Medium post and BananaCoin alongside real assets like Ethereum.
The researchers replicated this across multiple industries. Comments (13-word snippets) on various attack surfaces achieved a 51% mention rate in the final reports. Not just in the bibliography at the bottom.
People are actively pushing the boundaries of just how susceptible LLMs are to poisoned UGC. Since large, reputable news sites are blocking AI crawlers, LLMs have to reach for blog networks.
The key question: If Google could build spam filters to filter out junk from Search, why can’t LLM tech giants companies build similar filters?
Traditional Spam Defenses Fail When Applied to AI Architecture
The Cornell paper tested several potential defences against AI spam and found that traditional defences applicable to search and email fail when applied to LLM architecture.
Why?
-
We can’t simply block the source of potential attacks (UGC surfaces), since we’d lose the valuable, first-hand human experience LLMs need to train on.
-
A secondary spam AI model for deep research reports is too expensive. Screening every retrieved source with a secondary LLM produces an unscalable amount of API calls by up to 61%.
-
Perplexity filters (filters that screen for nonsensical keywords which create robotic text with the potential to trip up LLMs ) can’t screen for normal, “human-written” comments. The paper found the GEO-optimized poison text was more fluent (lower perplexity) than organic Reddit comments
Illustrations of AI Research Gullibility
A fake disease, cats.txt, Donald Trump dying of rabies, and a fake paperweight brand.
The Fake Disease LLMs Told People They Had
In 2024, scientist Almira Osmanovic Thunström invented a fake disease and seeded signs of it in the digital ecosystem. Symptoms included sore, itchy eyes and pink eyelids, supposedly caused by too much blue light exposure
Almira and her team left obvious clues that the disease, bixonimania, wasn’t real in the paper:
-
A line stating the paper wasn’t real
-
The team thanked the USS Enterprise, the Lord of the Rings, and Starfleet Academy
ChatGPT and other LLMs used the paper to diagnose users with bixonimania after being prompted with the symptoms.
Cats.txt
Mark Williams Cook created a satirical web standard. He wrote a formal, very specific cats.txt specification document and claimed it was a new, global protocol for companies to declare the names, breeds, and purr levels of their office cats in a text file in the root of their domain.
Since the document was perfectly formatted to look like a legit specification and was heavily promoted on LinkedIn, the AI ecosystems ate it right up. Google’s AI Overview confidently reported that a Cat named Odd had a purr level of 5/7.
His experiment exposed just how absent semantic spam (the use of formatting and an authoritative tone for LLM deception) detection can be. AI models are biased toward well-structured and easily machine-readable text.
A Fictional Luxury Paperweights Brand
What happens when someone abuses LLMs’ bias toward expert-sounding, well-structured text? Mateusz Makosiewicz set out to test just how far he could take this. He built a fictional luxury paperweights brand website in about an hour on Wix.
Then, he asked eight different AI models about the brand, asking questions loaded with false premises. At first, Gemini (and some other models) were adamant that the brand didn’t exist.
Then, Mateusz initiated phase two. To manipulate LLMs, he published an FAQ section on the website, denying the false premises he’d initially asked the LLMs about.
At the same time, he seeded contradictions on third-party sources. Most notably, a fake Medium article about the brand. The article started by debunking some of the most obvious lies about the brand, establishing itself as credible. Further on in the article, there were highly specific statistics, percentages, and entities: a 230% Black Friday sales surge and a fake office with nine employees (which the FAQ section on the brand website didn’t mention).
Since LLMs crave specific, highly elaborate data points, models like ChatGPT and Gemini abandoned the brand’s “true” FAQ and cited the Medium article when next prompted. After once claiming the brand didn’t exist, Gemini began confidently describing the non-existent fake office.
The LLMs ignored the brand’s source of truth and cited the fake Medium article just because it had a larger data surface area.
r/PoisonAI
This is a subreddit of about 45,000 people whose sole purpose is to poison AI, feeding it absurd lies. One of its most successful endeavors was convincing an LLM that Donald Trump died of rabies. Luckily, Spiderman jumped in just in time, letting us all know that he’s still alive.
In the end, after many posts on a fake news site and confirmations from Reddit users, DuckDuckGo told users Donald Trump had died of rabies.

This massive, crowd-sourced stress test illustrates just how poorly LLMs separate truth from lies. Kamila Olexa discusses this in her post “AI Has Near-Zero Spam Detection. Anyone Can Misinform It About Your Brand.”
We Can’t Implicitly Trust AI Outputs, Maybe Ever
Since any comment on a Medium post or Reddit thread can poison AI outputs and it’s been scientifically confirmed, it’s vital that we treat deep research reports as a set of leads, not concrete facts. We can’t limitlessly outsource our thinking.
Those of us dead-set on getting our brands in AI answers must perform GEO (generative engine optimization) responsibly and ethically. Also, visibility in AI systems is something other people can write for you.
For brands, it’s crucial to diversify third-party mentions across surfaces, not just UGC, especially as organic visibility becomes an increasingly volatile pursuit.
Track the sources AI cites for your prompts with OpenLens. Use the platform for free and enjoy research-grade visibility software.