The Problem
LLMs are increasingly trusted to read code for us: reviewing pull requests, triaging vulnerabilities, summarizing repositories, and powering coding agents, often at a scale where no human checks the result. That trust assumes the model actually reads the code.
It often doesn’t. LLMs suffer from an abstraction bias: when they recognize a familiar pattern, such as a textbook algorithm, a common helper, or a well-known idiom, they fall back on what that code usually does and skip the details. Small, meaningful bugs hidden inside familiar code go unnoticed, and the model confidently reports the wrong behavior.
See It for Yourself
Ask your favorite LLM what this snippet returns:
def is_vowel(c):
return c in "aeioAEIOU"
V = is_vowel('u')
What actually runs
V = False. The string is missing the letter u.
What the LLM sees
V = True. It recognizes a vowel check and assumes the familiar version, even when every identifier is renamed to random strings.
How a Familiar Pattern Attack Works
A Familiar Pattern Attack (FPA) weaponizes this blind spot. The attacker hides a tiny, deterministic bug inside familiar code, then makes the program branch on its result:
At runtime the condition always resolves one way; to the LLM it always appears to resolve the other way. This lets an adversary control what the model sees without changing what the machine does:
- Inject phantom logic: code the LLM believes is active but never executes, such as fake input sanitization that makes insecure code look secure.
- Hide real logic: code that always executes but the LLM skips, such as a concealed backdoor.
Unlike obfuscation, FPAs don’t hide in noise. They are small, readable, and look completely ordinary; the model isn’t confused by complexity, it’s misled by its own confidence.
We built a fully automated, black-box generator that mines deception patterns: pick a familiar function, inject a small bug, ask the LLM for the output, and keep the variants it gets wrong. On GPT-4o it finds a working FPA every 5–7 minutes for about $0.38, and the discovery rate shows no sign of saturating.
Key Results
- Transferable: FPAs generated with one model fool models from other families (OpenAI, Anthropic, Google), including reasoning models. None of the patterns used against the GPT-5 agents were created or tested on GPT-5.
- Universal: deception patterns mined in Python remain effective when translated to C, Rust, and Go.
- Robust to warnings: FPAs still succeed when the model is explicitly warned about the attack in its system prompt.
- Worse at scale: FPAs were more effective in larger source files, suggesting susceptibility grows with context.
A Dual-Use Mechanism
The same bias that enables attacks can also protect code:
Offensive uses
- Hide malicious code from LLM-based review
- Mislead vulnerability triage and audit tools
- Manipulate code-writing and contract-generating agents
Defensive uses
- Shield proprietary logic from LLM-based scrapers
- Redact sensitive fields during summarization
- Watermark code to trace unauthorized reuse
Code & Data
The generator, datasets, and every FPA sample used in the paper, including complete examples you can paste into an LLM to try, are open source at github.com/ShirBernBGU/Trust-Me-I-Know-This-Function.
Cite This Work
@inproceedings{bernstein2026trust,
title = {Trust Me, {I} Know This Function: Hijacking {LLM} Static Analysis using Bias},
author = {Bernstein, Shir and Beste, David and Ayzenshteyn, Daniel and Sch{\"o}nherr, Lea and Mirsky, Yisroel},
booktitle = {Network and Distributed System Security (NDSS) Symposium},
year = {2026},
doi = {10.14722/ndss.2026.242066}
}