Ask a coding agent to track down a bug and it will often tell you exactly what it did: updated the retry logic in src/lib/http/retryClient.ts, added a null check inside formatInvoiceTotal, imported the new helper from utils/currency. Go looking for that file and it does not exist. The method it called is not defined on that class. The import points at a module nobody ever wrote. The agent was not lying. It described a change to a codebase that exists only in its head, and it described it with the exact same flat confidence it uses for a change that actually happened.
This is not a rare glitch that a stricter system prompt will patch over. It is the default behavior of the thing doing the writing. A language model has no file system of its own. What it has is a training distribution and whatever text happens to be sitting in its context window at the moment it answers. When the real answer to "does this file exist" is not in either place, the model does not stop and check. It produces the most statistically plausible-sounding path it can construct, because producing plausible-sounding text is the only operation it was ever built to perform. Understanding why that happens, and why the fix is neither a bigger context window nor a more sternly worded prompt, means starting with what the model is actually doing when it names a file.
Origin: what the model is doing when it names a file
A large language model generates text one token at a time, each one chosen because it is the statistically likely continuation of everything that came before it, given the patterns the model absorbed during training. There is no separate step where it looks something up and checks whether the token it is about to produce corresponds to a real thing in the world. Fluency and accuracy are produced by the same mechanism, and nothing in that mechanism natively distinguishes between them.
OpenAI researchers Adam Tauman Kalai, Ofir Nachum, Santosh Vempala, and Edwin Zhang made this precise in a September 2025 paper, Why Language Models Hallucinate. Their argument is that hallucinations originate as ordinary errors in a kind of binary classification problem baked into next-token prediction: when the training data does not cleanly separate a plausible-looking wrong answer from a correct one, the statistical pressure to predict a likely token does not reliably tell them apart either. A fact that appears once, arbitrarily, in the training data (a person's birthday, a license plate number) is exactly the kind of thing a model will guess at with total confidence, because guessing is what minimized loss during training on everything else. The file layout of your private repository is that kind of fact, except worse. It was not in the training data at all.
That does not mean the model draws a blank. It means it does something less obviously wrong and more quietly dangerous: it generalizes. Across millions of public repositories, most Node projects keep helpers in src/utils/ and most Django apps have a models.py. The model has genuinely learned that pattern, and it is a good pattern to know. The failure is not that the model has no idea what a plausible file structure looks like. The failure is that it applies the modal, most common structure it has ever seen to your specific repository and states the result as an observation rather than a guess.
This is also why hallucinated references are not random noise you can average away by asking twice. A 2025 USENIX Security study on package hallucinations, covered in more detail below, re-ran the same failure-triggering prompts ten times each and found that 43 percent of hallucinated package names came back identically on every single run, and 58 percent repeated more than once. A model hallucinating at random would not produce the same wrong answer nine times out of ten. A model confidently generalizing a learned pattern will, because the pattern, not the fact, is what is actually driving the output.
Present: a flat context window is not a map
There is exactly one way for the model to get this right instead of pattern-matching its way to a plausible wrong answer: the true fact has to be sitting in its context window at the moment it answers. This is why agents read files, run searches, and retrieve documents before responding. It is also why doing that does not fully solve the problem.
A context window is not a database and not a map. It is a flat sequence of tokens, rebuilt fresh on every turn of an agent loop, with no structure beyond the order things were pasted in. If an agent reads retryClient.ts on step three, that file's contents become tokens sitting somewhere in a growing sequence alongside the system prompt, the user's request, four tool calls, and whatever the agent said to itself along the way. Nothing about that arrangement tells the model which of those tokens are verified ground truth about the repository and which are its own earlier speculation. Both look like text in the window. Both get attended to by the same mechanism.
Even when the correct information genuinely is present in context, position matters more than it should. Researchers behind Lost in the Middle, later published in the Transactions of the Association for Computational Linguistics, found that models handle information at the start and end of a long context well and lose accuracy on information buried in the middle, producing a U-shaped performance curve that holds even for models explicitly built for long contexts. A file's real export list, retrieved accurately on step two of a twenty-step agent run, is exactly the kind of fact that can end up in that middle stretch by the time it matters.
Then there is what happens as the loop keeps running. Long agent sessions get compacted or summarized to stay inside the window, a dynamic covered in more depth in context rot. A summary of "read currency.ts, it exports formatCurrency and parseCurrency" is a lossy compression of the original fact, and by the time an agent is twenty steps deep, it is often reasoning from a chain of compressed memories of what it once verified rather than the literal file. Being in context now is not the same thing as being verified. Once it scrolls out, or gets summarized away, the model is back to generalizing from its training distribution, which is where this problem started.
Present: imports, functions, and paths
The mechanism is one thing. The concrete ways it shows up in a coding agent's output are worth separating, because they carry different amounts of risk and different amounts of research attention.
Import and package hallucination is the best-measured version, because unlike a private repository's file layout, a package name can be checked against something public: the actual PyPI or npm registry. Joseph Spracklen and colleagues at three universities did exactly that at scale, generating 576,000 Python and JavaScript code samples across sixteen code-generating models and checking every recommended package against the real registries, in a paper analyzing package hallucinations across sixteen code-generating LLMs, presented at USENIX Security 2025. Overall, 19.7 percent of recommended packages did not exist. Open-source models hallucinated far more, 21.7 percent on average, than commercial ones at 5.2 percent, and across the whole study the researchers logged more than 205,000 unique hallucinated package names.
That number matters beyond wasted time, because a hallucinated package name is an attack surface. If a model reliably suggests installing a package that does not exist, an attacker can register that exact name and wait. Seth Larson, security developer in residence at the Python Software Foundation, coined a name for it in April 2025: slopsquatting, a portmanteau of AI slop and typosquatting. Socket's analysis of the same dataset found the repetition consistent enough that squatting on the popular hallucinated names is a viable strategy, not a lottery ticket. A developer, or an increasingly autonomous agent, that trusts the suggestion and runs the install command has just pulled arbitrary code from a stranger.
Function and method hallucination follows the identical mechanism aimed at an API surface instead of a package index. An agent asked to check a subscription status might write user.getActiveSubscription() because dozens of SDKs it trained on shape their API that way, when the actual class exposes user.subscriptions().active(). Nothing about generating that call feels different, internally, from generating a call that is correct. It is the same generalization move described above, just aimed at a method signature instead of a file tree.
File path hallucination is the least studied of the three, and that is worth sitting with rather than reading as reassurance. Package names are checkable against a public registry, which is exactly why they have a peer-reviewed study with a precise percentage attached to them. There is no equivalent global index of every file path in every private codebase, and the research literature on it is thinner as a direct result. The absence of a benchmark number is a gap in what is easy to study, not evidence that it happens less. In practice it looks like an agent asserting it edited a file that was never created, or confidently assuming a project uses one folder convention when this particular one uses another, or inventing a tests/ directory in a codebase that calls it spec/. In an agentic loop where later steps build on earlier ones without a human reading every diff, a fabricated file path from step four can quietly steer steps five through fifteen before anyone notices nothing was ever there.
Why bigger context windows and better prompts only shrink the problem
The obvious fix is to give the model more room and more explicit instructions. Both help. Neither closes the gap, and it is worth being specific about why.
A bigger context window sounds like it should solve this by letting an agent paste in more of the actual repository. The trouble is the mechanical one from the previous section: recall degrades well before a window is anywhere near full. Chroma's 2025 context rot research tested eighteen frontier models and found performance sagging steadily as input length grows, often long before the advertised limit, so a window sold at hundreds of thousands of tokens can show real degradation at a small fraction of that. A bigger window is a bigger room to search, not a more reliable memory of what is in it. And in practice, most agent tasks do not paste the whole repository anyway. They retrieve slices of it through search or file reads, and a bigger window does nothing for a retrieval step that pulled in the wrong slice, or a merely similar one, in the first place.
Better prompting genuinely helps at the margin, and it would be dishonest to claim otherwise. Telling a model to only reference files it has actually read, or to say when it is uncertain, measurably reduces confident fabrication. But it is an instruction aimed at the output layer, not the cause. The model is still producing its best guess about how much confidence to project, using the same statistical machinery that produced the fabrication in the first place, just with one more instruction folded into the text it is completing. Kalai and his OpenAI coauthors make the sharper version of this point: language models are tuned against benchmarks that score a confident wrong answer the same as, or better than, an honest "I don't know," an incentive structure that rewards guessing over abstaining. A model shaped by that incentive does not stop being a guesser because a system prompt asked it nicely. Prompting can shift where the model draws the line between confident and hedged. It cannot hand the model a fact it has no way to check.
The clearest evidence that neither lever fully closes the gap comes from a setting engineered to remove the excuses. Vectara's hallucination leaderboard scores models on a single, deliberately easy task: summarize a short document faithfully, with the source document placed directly in the context window, nothing to retrieve, nothing to remember from ten steps ago. Even here, hallucination rates vary by well over an order of magnitude across models, and no model scores zero. If handing the model the correct answer on a plate does not eliminate fabrication, no amount of window expansion or prompt tuning is going to eliminate it either, because the window and the prompt were never the actual cause. The cause is that the model has no built-in way to distinguish a verified fact from a fluent guess, and nothing about a longer window or a firmer instruction changes that.
Grounding retrieval in a graph instead of a guess
If the cause is that the model cannot tell a verified fact from a fluent guess, the fix has to change what checking means, not how hard the model tries. That means looking at where an agent's answer about your repository actually comes from, and there are three meaningfully different sources.
The first is the model's own memory, its training distribution. This is what produces the fabrication described throughout this piece. It carries no factual guarantee about your specific codebase at all, because your codebase, by definition, is not the thing the model memorized.
The second is retrieval by similarity, the vector-search approach behind most retrieval-augmented generation systems. This is a real improvement, because the agent is now looking at your actual code instead of pure recollection. But similarity is not the same property as correctness. An embedding search for "the function that formats currency" returns whichever chunks of text sit closest to that query in vector space, which might be the real function, a similarly named one in a different module, or nothing useful if the phrasing does not line up well with how the code was written. Crucially, a similarity search has no way to return a confident negative. Ask it whether formatCurrency exists in utils/currency.ts and it can only ever hand back the closest thing it found. It cannot say the path is not there, because closest match is the only kind of answer the method is built to produce.
The third source is a structural graph, built by actually parsing the codebase rather than embedding it: the real files, the real imports, the real call relationships, extracted deterministically rather than approximated by semantic similarity. This changes the shape of the question. Whether formatCurrency exists in utils/currency.ts stops being a similarity-ranked guess and becomes a lookup against a graph that either contains that edge or does not. That is the actual mechanism behind the claim that grounding retrieval this way removes the cause rather than treating the symptom. A longer context window or a more careful prompt can only make the guess more careful. A verified graph can replace the guess with an answer, including the honest negative one, which is precisely the response pattern completion has no native way to produce on its own, since it only knows degrees of plausibility, never a clean absence.
This direction has independent backing in the research literature, separate from any single product. CodexGraph, built to connect LLM agents to a graph database constructed from a real code repository, was motivated by exactly this gap: similarity-based retrieval has low recall on complex, repository-scale tasks, so the authors let the agent issue structured queries against actual code relationships instead of matching on text similarity, and validated the approach on benchmarks including CrossCodeEval, SWE-bench, and EvoCodeBench. None of this makes an agent infallible. Reasoning can still go wrong, and requirements can still be misread. But the specific failure mode this piece is about, an agent inventing a file, an import, or a function that was never there, stops being something the model has to guess about, because the question of whether it exists no longer depends on the model at all.
Future and impact: what changes when an agent can say it is not there
The stakes here scale with autonomy. When a model only suggests a diff for a human to read, a hallucinated file path is caught in review and costs a few seconds of irritation. As agents move toward making multi-step changes on their own, running builds and opening pull requests without a human reading every intermediate step, a single fabricated reference stops being a cosmetic error. It becomes the false premise every later step is quietly built on: a broken build, a CI run burned on a function that never existed, in the worst case a slopsquatted package pulled into a dependency tree because nobody double-checked a name that sounded exactly right. The more independently an agent is allowed to act, the more expensive it gets for "I don't know" to not be a state the system can reach.
This is the specific problem Spiderbrain, built by Perform Digital Private Limited, is designed around. It parses a codebase locally, on the developer's own machine, and builds a deterministic, source-free context graph, what the product calls a brain, of the project's actual files, imports, and call relationships. Deterministic means the same codebase produces the same graph every time, a property verified by roughly 350 automated checks that run green on every release. Only the structural map and its scores leave the machine, sent to Spiderbrain's EU-based servers for scoring. The source code itself never does. That graph is exposed to coding agents over MCP (Model Context Protocol), the standard for connecting a model to external tools and data, and works with more than ten MCP clients, including Claude, Claude Code, Cursor, Continue, Cody, and Zed, alongside blast-radius analysis that shows what a change is likely to break before it ships. The point that matters for this piece specifically: because retrieval is grounded in a graph actually built by parsing the repository, rather than a similarity search over embeddings or the model's own memory, an agent connected to it can be told whether a file, an import, or a function genuinely exists, instead of being left to fill the gap with its best guess. For anyone curious about the mechanics, how a deterministic context graph works goes into the parsing and scoring pipeline in more detail.
None of this is a claim that hallucination becomes impossible. It is a claim about where the guessing happens. Right now, most coding agents are asked to be careful guessers: read more context, hedge more often, cite your sources. That helps at the margin and it was always going to plateau, because a guess dressed up in more caution is still a guess. The more durable fix is structural. Give the agent a verified map of what actually exists in the repository it is working in, and the question of whether a file, an import, or a function is real stops being a statistical judgment call made under uncertainty. It becomes something the system can simply look up.
Council summary
This post argues that AI coding agents invent file paths, imports, and functions because the underlying model has no mechanism for distinguishing a verified fact about your repository from a fluent, statistically plausible guess, and that neither a bigger context window nor a more careful prompt fixes the root cause because both operate on the guess rather than replacing it. It backs the claim with real, checkable numbers rather than general complaint: the USENIX Security 2025 finding that code-generating models hallucinate nonexistent packages in 19.7 percent of recommendations, the Lost in the Middle and Chroma context rot research showing recall degrades well before a window is full, and OpenAI's own explanation of why training and evaluation incentives reward confident guessing over honest uncertainty. It is honest about the limits of its own fix, noting that grounding retrieval in a parsed, deterministic dependency graph removes this specific failure mode rather than making an agent infallible in every other way. The reader's takeaway is a sharper question to ask about any coding agent setup: when it names a file, is that name coming from a verified structure, or from a guess wearing a confident voice.
Comments