Ask an AI coding assistant to find the function that retries a failed webhook delivery, and a vector-search system will probably get you there in one try. Ask the same assistant what happens if you change that function's signature, who calls it, and whether any caller assumes it never throws, and the answer comes back just as confidently. It is not the same kind of question. Retrieval-augmented generation and a structural code graph are two different answers to what does this codebase do, and a system built to answer one well is not automatically equipped to answer the other.
This site has already traced retrieval-augmented generation's general arc, from a fixed retrieve-then-generate pipeline to an agentic loop that decides when and how to search, and weighed it against fine-tuning and prompting as one of three answers to a knowledge problem. Code is a narrower case than either piece covers, and a genuinely different one. A codebase is not a pile of text that happens to be about a topic; underneath the text sits an actual, checkable structure, functions calling functions, modules importing modules, types depended on by code that breaks if they change. The question here is whether a retrieval method can see that structure, or only the words sitting on top of it.
Origin: two lineages that never had to compete until now
Code search split into two families early, for an unglamorous reason: they solved different problems and nobody needed them to talk to each other.
The first family is lexical and structural. grep and ctags go back to Unix in the 1970s: grep finds a literal string, ctags builds an index of where symbols are defined so an editor can jump to them, and cscope extended that into finding callers, not just definitions. This lineage matured into the Language Server Protocol, which Microsoft published in 2016 to standardize what a good editor already wanted to do: go to a definition, find every reference to a symbol, walk a call hierarchy. None of this is guessing. A language server parses the code with a real grammar for the language and answers from that parse. If it says a function has fourteen callers, there are fourteen callers, not fourteen likely candidates. This protocol's second act, letting AI agents reach tools through a standard interface, is covered in MCP: the USB-C port for AI agents.
The second family is semantic, and it arrived decades later, riding on general-purpose retrieval-augmented generation. Once embeddings made it cheap to turn any chunk of text into a point in vector space, and once the 2020 RAG paper from Lewis and colleagues at Facebook AI Research established retrieve-then-generate as the default way to ground a language model in an external source, teams pointed that same machinery at source code with barely any modification. Chunk the files, embed the chunks, index the vectors, retrieve by similarity at query time. It worked, in the sense that it retrieved code that read like the question, and that was enough to make it the default for the first wave of AI coding assistants.
Nobody had much reason to compare the two approaches directly, because they served different audiences. A structural index served an editor and a human clicking around it. A vector index served a chatbot answering a question in prose. AI coding agents collapsed that distinction. The same agent now has to answer "where do we already handle rate limiting" and "what breaks if I change this interface" inside one session, and only one of those is a resemblance problem.
Present: what vector RAG over a codebase is genuinely good at
Judge vector RAG by the question it was built for and it holds up well. Embed a codebase's functions, classes, and docstrings, and a query like "where do we already validate an email address" or "is there existing code that resembles rate limiting" will usually surface the right neighborhood, because that kind of question really is asking about resemblance: does something like this exist, expressed close to how the question just described it. Well-commented code, meaningful names, and docstrings all give an embedding model real signal, because they already read close to prose.
It is also the cheaper build by a wide margin. Standing up a vector index over a repository takes an embedding model, a vector store, and a chunking strategy, all commodity infrastructure now. It does not require a parser for the language, or much awareness that the text is code rather than prose; the same pipeline that indexes a company's documentation can index its source files with a light adjustment to chunking. That uniformity helps in a mixed repository, where one question might span a README, a design doc, and an implementation.
It also degrades more gracefully across half-finished or unusual code than a structural index can. A vector index does not care whether a file currently parses; it will embed a function sitting next to a syntax error without complaint. A structural index built by actually parsing the code has no such tolerance: if the parser cannot make sense of a file, that file contributes nothing to the graph until it is fixed. For a codebase mid-refactor, written in an unusual language, or partly generated, that tolerance is worth something.
Repository-level benchmarks back up the basic value of retrieval even when it is imprecise. RepoBench, the 2023 benchmark for repository-level code completion, found that pulling in cross-file context improves completion accuracy over working from the current file alone, and that the benefit shows up even when the retrieval is not especially well targeted: more of the surrounding repository, imprecisely selected, still measurably beats none. Approximate context beats no context, and for a large share of what a developer or an agent actually asks, approximate is good enough.
Present: where similarity stops being structure
The trouble starts the moment a question stops being about resemblance. Ask a vector-RAG-backed assistant what calls parse_config, and here is what happens: the question gets embedded, the system searches for chunks whose vectors sit closest to it, and hands back the chunks that mention parse_config the most, the definition itself, a comment, maybe a test file repeating the name. What was actually asked for, the real call sites, especially ones two or three hops away through a wrapper, is different information, and nothing guarantees it made the shortlist. FalkorDB, a graph database vendor that sells directly into this problem, states the failure plainly: ask about the blast radius of a refactor, and the assistant retrieves three plausible-looking files while missing the transitive caller two hops away, confidently.
The reason is not a bug in any particular implementation. It is what similarity search is built to measure. A function's place in a codebase is defined relationally: what calls it, what it calls, what it imports, what inherits from it, what breaks if its return type changes. None of that has to be present in the function's own text. Two functions can read almost identically and have nothing to do with each other at runtime; two functions can be worded nothing alike and be tightly coupled through a call chain. Cosine distance between two embeddings measures how alike two pieces of text read. It does not measure, and was never built to measure, whether one piece of code actually reaches the other when the program runs. A 2025 survey of retrieval-augmented code generation makes almost exactly this the organizing question of the field, sorting the whole landscape of repository-level approaches into graph-based and non-graph-based retrieval paradigms. That framing is itself a tell: enough of the field now treats structural understanding as what separates one code-RAG system from another.
The failure compounds because chunking, the step every vector pipeline needs to fit code into an embedding model, cuts through exactly the boundaries that carry structural meaning. A function gets embedded without the class it belongs to; a call site gets embedded without the import that resolves it. CodeRAG-Bench, the benchmark built specifically to test whether retrieval helps code generation, found that retrievers still struggle to fetch useful context when there is limited lexical overlap between a question and its answer, exactly the situation a structural question creates, since "what calls this" rarely shares vocabulary with its callers. A separate study of what to actually retrieve for code generation found that retrieving code merely because it looks similar can actively hurt: noisy, similar-but-irrelevant snippets degraded generation quality by up to 15 percent, while grounding the model in real in-context code and API information, closer to structural fact than resemblance, helped substantially more. More retrieval is not automatically more signal, and the wrong kind is worse than none.
The sharpest version of this failure shows up as a security problem, not just a wrong answer. A model with no ground truth about which packages, functions, or files actually exist will produce a plausible invention when it does not know the real one, and it does this at scale. The most-cited study of the phenomenon tested 16 large language models across 576,000 code samples and found that a meaningful share of recommended packages did not exist at all, at least 5.2 percent of the time for commercial models and 21.7 percent for open-source ones. Attackers now register those exact invented names ahead of time, a technique researchers call slopsquatting, so a model's confident guess becomes an installable, malicious package. Why agents invent file paths in the first place is its own subject; the point here is narrower, that this is what happens anywhere a system answers a structural question, does this dependency exist, from resemblance instead of ground truth.
Even outside strictly structural questions, plain similarity search has been losing ground in production code tools, for reasons adjacent to this one. Anthropic's own Claude Code shipped early versions with a local vector database, then removed it. Its creator, Boris Cherny, told journalist Gergely Orosz that plain glob and grep, driven by the model, beat everything the team tried, including the vector pipeline, by enough to be surprising. The honest caveat is that this result is about lexical search beating vector search on general code navigation, not proof that a structural graph is the answer to everything. But it points the same direction as the rest of the evidence here: a method that respects what is actually written or connected keeps outperforming one that only measures resemblance.
Present: what a structural graph gets you, and what it costs to keep one
A structural code graph inverts the approach, and it is worth being precise about what "graph" means here, since the word also gets used for something built by an LLM guessing at relationships in prose rather than parsing them, a distinction with real consequences. This kind of graph starts by parsing the code with a real grammar for the language, the way a compiler or language server does, and records what it finds: a node for every function, file, class, and type, an edge for every call, import, inheritance, and reference, each edge typed and directional. Once that graph exists, "what calls parse_config" stops being a search problem and becomes a traversal problem: walk the incoming call edges. "What is the blast radius of changing this type," a computation worth walking through in detail, is a walk outward along every edge that references it, as many hops as the question needs. The answer is exhaustive and deterministic rather than a ranked guess, because it comes from the program's actual structure rather than from how closely two pieces of text read.
This is precisely the distinction Sourcegraph, a code-search company, draws between its embeddings-based search and its structural index, SCIP: embeddings are good at a broad, exploratory question and lose usefulness once the question narrows to a specific implementation detail, while a precise structural index returns every reference to a symbol, not a ranked list of likely matches. When a question has a factually correct, enumerable answer, a graph gives you that answer. When the question is really "does anything like this exist," a graph gives you nothing, because it has no notion of resemblance at all. Two functions can sit next to each other in a call graph and read nothing alike, and a pure structural index will not connect them to a fuzzy conceptual question the way an embedding will.
None of this is free, and the honest accounting matters as much as the capability. Building the graph means running an actual parser over every file in every language the repository uses, not a generic embedding model that can shrug at a syntax error. That is real engineering per language, and it is brittle in a way a vector index is not: a partial or unparseable file contributes nothing to the graph, where it would still contribute something, however fuzzy, to an embedding index. Freshness is the harder problem. A vector index that is a day stale returns slightly worse-ranked results. A dependency graph that is a day stale can assert a call chain that no longer exists, or miss a new one entirely, and hand that wrong structural claim to a model with the same confidence as a correct one. Keeping a graph honestly synchronized with a codebase that changes every commit is more work than re-embedding a changed file, and it has to run continuously, not once.
A graph is also not a strict upgrade on every kind of question, and a study on building tree-sitter-based knowledge graphs for LLM code exploration is unusually candid about the trade-off. Its graph-native retrieval used roughly ten times fewer tokens and about half the tool calls of a plain file-exploration agent, and matched or beat file exploration on structural queries, hub detection and caller ranking, in nineteen of thirty-one languages tested. But on general, open-ended questions, the ones closer to "does something like this exist" than to "what calls this," the graph-native approach scored lower than plain exploration: 83 percent answer quality against 92 percent. A graph is a better tool for a structural question and a worse one for a resemblance question, in the same study, on the same codebases. That is the honest shape of the trade-off, not a story where one approach simply wins.
Future and impact: routing the question to the tool that can answer it
None of this argues for retiring either approach, and it would be a strange argument to make, because the two failure modes are mirror images of each other. A vector index answers a structural question with a confident guess dressed up as a citation. A structural graph answers a resemblance question with nothing at all, because "does anything like this exist" has no edges to traverse. The realistic conclusion is close to what this site already found true for knowledge graphs generally: graph and vector retrieval solve different problems well enough that serious systems increasingly run both and route each question to the one suited for it, rather than betting a whole architecture on a single mechanism. Sourcegraph's own assistant already blends code search, SCIP graph traversal, and vector embeddings behind one interface, because no single retrieval strategy covers both a conceptual question and a structural one.
What is changing is who does the routing. The earlier pattern was a fixed pipeline: a team picked vector search or a structural index up front and built the whole assistant around that choice. The shift this site traced through agentic RAG, an agent deciding when and how to retrieve rather than following one fixed step, applies here with almost no modification. Give an agent both a similarity search and a graph traversal as tools, and it can send "where do we handle this" to the vector index and "what calls this" to the graph, inside the same session, without a human pre-classifying it first. That does not remove the underlying trade-off. An agent with only similarity search and no graph to traverse can behave more carefully around a structural question, cross-checking itself, issuing more tool calls, hedging its answer, but it is still approximating a structural fact from resemblance, at higher cost in time and tokens, with no guarantee of completeness. Having a graph available changes what the agent can actually know, not just how confident it sounds while guessing.
One example of this pairing worth naming directly, because it is built by the same company publishing this piece. Spiderbrain builds the structural-graph side of it: a deterministic map the company calls a brain, covering a codebase's functions, files, and dependencies, engineered so the same codebase produces the same graph every time, with roughly 350 automated checks run green on every release. Parsing happens locally on the developer's own machine, and only a source-free map of structure and scores, not the source code itself, goes to Spiderbrain's EU-based servers for scoring, which matters because a structurally useful graph is also detailed enough to leak the codebase it describes if handled carelessly. On top of that graph sit blast-radius analysis, a real call and dependency graph, and memory that persists across sessions instead of resetting with the context window, the same problem as the package hallucinations cited earlier: an agent that can check a claim against a graph instead of guessing from resemblance is less likely to invent a file path or a dependency that was never there. It speaks MCP and connects to ten or more clients, including Claude, Claude Code, Cursor, Continue, Cody, and Zed, so it sits alongside whatever retrieval an agent already does rather than replacing it. How the graph gets built is its own piece; the point here is narrower, that taking the deterministic side of this trade-off seriously is a real engineering commitment, not a checkbox.
Pick between these two the way you would pick any tool: by the shape of the question, not by whichever one is already running. If the question is "does something like this exist," start with similarity search. It is cheap to stand up, it tolerates half-finished and unfamiliar code, and it will usually get you to the right neighborhood. If the question is "what calls this," "what breaks if I change it," or "what is the real dependency order here," similarity search will give an answer that sounds plausible without being correct, and the honest move is to reach for a graph and accept the extra cost of building and refreshing it, because a wrong structural claim stated with confidence is worse than an honest "not sure." Most real codebases need both kinds of question answered in the same afternoon. That is the argument for pairing the two, not for crowning one.
Council summary
This piece argues that RAG and a structural code graph answer two different classes of question about a codebase, resemblance and structure, and that neither is a lesser version of the other. It grounds the RAG side in real strengths: cheap setup, language-agnostic chunking, and RepoBench's finding that even imprecise cross-file context beats none, before showing where similarity search structurally cannot follow, from CodeRAG-Bench's retrieval gap on low-lexical-overlap queries to a repository-level survey that now organizes the whole field around graph versus non-graph retrieval, a measured 15 percent quality hit from noisy similar-code retrieval, and a documented pattern of package hallucination that turns the same blind spot into a supply-chain risk serious enough to have its own name. It grounds the graph side just as honestly, citing a study where graph-native retrieval used a tenth of the tokens and won on structural queries but lost to plain file exploration on open-ended ones, 83 percent to 92, which keeps the piece from overselling either tool. The reader's takeaway is diagnostic, not partisan: work out what kind of question is actually being asked before choosing how to answer it, reach for similarity search when the question is whether something exists, reach for a graph when the question is what depends on it, and expect the honest answer, increasingly, to be both, routed by an agent rather than committed to in advance.
Comments