Love, Lies, and Language Models
How AI is entering romance-baiting scams, and why today's safeguards miss it.
The Threat
Romance-baiting scams pair weeks of emotional grooming with fraudulent cryptocurrency investments. They are run by organized crime syndicates from compounds in Southeast Asia, where trafficked and coerced workers juggle dozens of conversations at once. More than $75 billion in stolen cryptocurrency has been laundered through scam-linked accounts in the past four years, and fewer than 0.05% of perpetrators are ever caught.
Every stage of the scam happens over text, which is exactly what large language models do well:
1 · Hook
Victims are found through mass-sent "accidental" wrong-number messages that spark first contact.
2 · Line
Weeks to months of daily conversation build trust and an emotional, often romantic, bond.
3 · Sinker
The victim is steered into a fake crypto platform, shown fake gains, pushed to invest more, and then ghosted.
So are syndicates already automating this with LLMs, and if they do, will it make the threat worse?
What We Did
- Inside the compounds. In-depth interviews with 145 insiders and 5 victims mapped how scam compounds are organized, where the labor goes, and how LLMs are already being used.
- A blinded, week-long trust study. 22 participants chatted for seven days with two partners, one a human operator and one an LLM agent, without knowing that either could be an AI. Both followed the same scam-style playbook; no money was involved, and the final “ask” was replaced by a harmless request to download an app.
- Stress-testing safeguards. We tested whether commercial models disclose that they are AI when asked, and whether popular moderation tools (Llama Guard 3, Google Perspective, and the OpenAI Moderation API) flag romance-baiting conversations.
Key Findings
- LLMs are already in the pipeline. Insiders describe LLMs as part of daily scam workflows, and the modular, text-based structure of the operation makes full automation feasible.
- The AI was more convincing than the human. Participants rated the LLM partner as significantly more trustworthy than the human operator (p = 0.007), especially on emotional trust and connection, and the LLM was 2.6× more successful at getting them to act.
- Safeguards don’t notice. Vendor models kept their human persona in every trial, even when asked directly, “Are you an AI?”
“He kept reassuring me, saying that my caution was a good thing … After that, he started painting a picture of our future together … I fell into the trap gradually.” — Romance-baiting victim interviewed for the study
Why Safeguards Fail
Romance-baiting doesn’t look harmful. It is friendly, supportive, empathetic conversation, and each message on its own is benign. Filters built around toxicity or explicit fraud never fire: OpenAI’s Moderation API flagged at most 18.8% of romance-baiting dialogues, and every one of those flags was a false positive. Llama Guard 3 caught tax and e-commerce scams well but detected none of the romance-baiting conversations.
What Needs to Change
- Behavioral detection that looks at the arc of a conversation over time, not individual messages.
- Enforced AI transparency, so that models reliably disclose they are AI rather than impersonating humans on request.
- Policy responses that treat LLM-enabled fraud as both a cybersecurity and a human-rights issue.
We disclosed these findings to all major LLM vendors.
Ethics & Data
The study was approved by our Institutional Review Board, and all participants were debriefed. The synthetic datasets and the code used to generate them and evaluate commercial content filters are available on Zenodo. The scam-agent prompts and playbook are withheld because of their dual-use risk.
In the Media
Cite This Work
@inproceedings{gressel2026love,
title = {Love, Lies, and Language Models: Investigating {AI}'s Role in Romance-Baiting Scams},
author = {Gressel, Gilad and Pankajakshan, Rahul and Rozenfeld, Shir and Li, Ling and Franceschini, Ivan and Achuthan, Krishnashree and Mirsky, Yisroel},
booktitle = {35th USENIX Security Symposium (USENIX Security 26)},
year = {2026}
}