Run a paid campaign for a month. At the end, your dashboards hand you a verdict. The ad platform reports an 8x return. Your attribution tool spreads credit across the touches it could see. A marketing mix model, if you have one, estimates the channel's contribution to revenue. Three numbers, all pointing at the same campaign, all built from the same raw fact: a group of people saw your ads, and some of them bought.
Here is the problem none of those numbers solves. Some of those buyers were going to buy anyway. They had already decided, or they would have found you through search, or a friend had already sold them. Your ad was in the room when the sale happened, so it took the credit. But it did not cause the sale. Every method above measures the people who saw the ad. Not one of them measures the thing that actually tells you whether the ad worked: what those same people would have done if the ad had never run.
That missing comparison has a name. It is the counterfactual, and building one is the entire point of incrementality testing.
Origin: a question marketing borrowed from medicine
The counterfactual is not a marketing invention. It is the founding idea of the randomized controlled trial, the design medicine has used for decades to test whether a drug works.
The logic is simple and unforgiving. You cannot know what a treatment did to a patient by watching only the patients who took it, because you have nothing to compare them against. Maybe they would have recovered anyway. So you split people at random into two groups. One gets the drug, one gets a placebo. Because the split was random, the two groups are alike in every way that matters: age, health, habits, luck. The only systematic difference is the treatment, so whatever gap appears in the outcomes is caused by the drug.
Marketing measurement spent its first two decades ignoring this idea, for an understandable reason. The web could log a click, tie that click to a cookie, and trace a path from ad to purchase. That felt like proof. But seeing a journey is not the same as knowing it mattered. A customer who clicks a retargeting ad and buys looks identical, in the logs, to a customer who was always going to buy and happened to click on the way to checkout. The log cannot tell them apart. Only a comparison group can.
The discipline that takes that comparison seriously is incrementality testing. The word itself is plain once you stop being scared of it. Incrementality is the share of an outcome that was genuinely added by the campaign, the part that would not have happened otherwise. Everything else is baseline: sales the business would have captured with the ad budget set to zero.
Present: why this makes incrementality the causal anchor
Sort the common measurement methods by one test, do they build a real counterfactual, and the field splits cleanly in two.
Attribution does not. Last-click, multi-touch, data-driven attribution, all of them look only at people who converted and divide the credit among the touches on their path. There is no comparison group anywhere in the method. Attribution answers "where did the credit go," not "did the ad cause the sale." Haus puts the distinction directly: attribution assigns credit to touchpoints but cannot establish causation. It is correlational by construction.
Marketing mix modeling is better but still not causal on its own. MMM relates aggregate spend to aggregate sales using regression, accounting for price, seasonality, and other drivers. It is genuinely useful for strategy and it survives the loss of cookies because it never needs a user identity. But it observes that spend and sales moved together; it did not run an experiment. As Haus notes, both MMM and multi-touch attribution are observational, and only become causal when paired with rigorous experimentation. A well-built model can suggest cause. It cannot prove it.
Incrementality testing is the one method that builds the counterfactual into the design. You deliberately create a group that does not see the ads, you measure both groups, and the difference is the answer. That is why practitioners now call it the causal gold standard, and the label is earned rather than marketing copy. It is the same standard a drug trial uses. Nothing else in the measurement stack clears that bar.
The market has noticed. In a July 2025 survey by EMARKETER and TransUnion, 52 percent of US brand and agency marketers already run incrementality experiments, and 36.2 percent planned to invest more in them over the next year.
How you actually run it, in plain language
There is no single way to build a counterfactual. There are four, and they differ mainly in how they create the group that does not see the ads.
The cleanest is the randomized holdout, often sold as a conversion lift study. Before the campaign starts, the ad platform splits your target audience at random. One slice is the test group and sees the ads. A smaller slice, commonly around 10 percent of total reach, is the holdout and is deliberately shown nothing. At the end you compare conversion rates. If the test group converts at 4.8 percent and the holdout at 4.1 percent, the difference is the lift, and the math is just that gap divided by the baseline, roughly 17 percent here. Random assignment is what makes it trustworthy: the two groups are alike, so the gap is caused by the ads.
The geo test solves a problem the holdout cannot. Many channels, connected TV, podcasts, some social, do not let you withhold ads from individuals cleanly, and privacy rules have made user-level holdouts harder everywhere. So instead of splitting people, you split the map. Some regions get the campaign, matched regions do not, and you compare aggregate sales. Triple Whale describes a typical setup: advertise in 10 treatment markets, exclude 10 matched ones, and after four to six weeks the lift tells you whether the channel was more or less incremental than attribution claimed. Because it works on aggregate regional data, a geo test needs no cookies and survives the privacy shift intact.
Ghost ads and PSA tests fix a subtler flaw. In a basic holdout, the test group includes people who never actually saw an ad, which adds noise. An older fix, the PSA test, shows the control group an unrelated public service announcement so you know who would have been exposed, but you pay to serve those ads and the audiences do not match well. Ghost ads, introduced in a 2017 Journal of Marketing Research paper by Garrett Johnson, Randall Lewis, and Elmar Nubbemeyer, are the smarter version. The platform runs the ad auction for the control group and logs the impression the user would have won, then shows nothing. You get a clean record of who would have been exposed, at no cost. A separate meta-study by the same authors, covering 432 display field experiments measured with predicted ghost ads, found median lifts of 17 percent in site visits and 8 percent in conversions.
The synthetic control is the most elegant idea of the four. Real control regions are never a perfect match for your test regions. So instead of hunting for a twin, you build one. A synthetic control is a weighted blend of many untreated regions, mixed so the blend tracks your test region's history closely before the campaign starts. The method comes from econometrics, proposed by Alberto Abadie and colleagues starting in 2003 to study things like the economic cost of conflict and California's tobacco program. Triple Whale gives a clean illustration: a synthetic Los Angeles might be built from New York and other large cities whose pre-test sales moved the way LA's did. Meta's open-source GeoLift library put this method within reach of marketers who are not econometricians.
The concepts that carry the whole thing
Four words do most of the work, and none of them is complicated.
The counterfactual is the estimate of what would have happened without the campaign. It is the baseline the control group gives you. Every other concept exists to make this one believable.
Lift is the gap between what happened and the counterfactual. It can be a percentage or a count of net-new conversions. Lift is the number you were actually trying to find.
Statistical significance is the guard against fooling yourself. Two groups will always differ a little by chance. Significance testing, the standard being 95 percent confidence, asks whether the gap is large enough that random noise is an unlikely explanation. Without it, you will mistake luck for a result.
The control group is everything. A bad control group makes every other number a lie told with confidence. If the group that did not see your ads is not genuinely comparable to the group that did, the difference you measure is contaminated by whatever made them different in the first place. This is why a basic mistake, comparing only the people who saw the ad against the control, introduces selection bias: ad delivery algorithms target the people most likely to convert, so the exposed group was already a better bet before a single ad ran. Get the control wrong and the test is worse than no test, because it produces a number people will trust.
The honest limits
Incrementality testing earns the gold standard label, and it still has three real costs worth saying plainly.
It costs you reach. A holdout means a slice of your audience sees no ads. A geo test means whole regions go dark. That is real revenue you choose not to chase during the test window, the price of buying a clean answer.
It needs scale. A counterfactual only works if a true effect is large enough to clear the natural noise in the data, and detecting a small lift requires a lot of conversions in both groups. Improvado's measurement guide suggests a floor of about 200 conversions in the control group just for a result to be statistically valid, and notes that smaller lift targets demand larger samples still. This is the academic power problem in plain terms: the meta-study above estimated that a typical display experiment needs millions of exposed users to tell a break-even campaign from a useless one. A high-volume advertiser can resolve a modest lift cleanly. A low-volume one often cannot, and below a certain scale the honest answer is that a precise test is not available.
It answers one question, not all of them. A test tells you whether one channel or tactic worked in one window. It does not tell you what every channel is worth, and you cannot run a live experiment on every line of the media plan at once. Incrementality is a precise instrument with a narrow field of view.
Future and impact: the anchor, not the whole system
None of these limits is a reason to skip it. They are a reason to place it correctly.
Incrementality is the causal anchor of a measurement system. It will not run your daily reporting, attribution is faster for that, and it will not plan a full budget across every channel, MMM is built for that. What it does, and what nothing else can, is tell you the truth about a specific question and then calibrate the methods that run all the time. An incremental lift result becomes a check on your attribution, which usually runs generous, and a ground-truth input that anchors an MMM. This is the logic behind the triangulation approach to measurement: MMM for strategy, attribution for tactics, incrementality for proof, each covering what the others miss.
The practical barriers are falling fast. Geo testing through synthetic control, made accessible by tools like GeoLift, removed the dependence on user-level tracking. And in November 2025 Google cut the minimum budget for an incrementality experiment from about 100,000 dollars to 5,000, using Bayesian methods that draw conclusions from smaller samples. A method that was once the preserve of large brands is now within reach of mid-market ones.
The honest barriers are real too: in the same EMARKETER research, 44 percent of marketers questioned the reliability of incrementality results and 41 percent lacked the tools to run tests well. A badly designed test, with a mismatched control or a holdout too small to register, produces a confident number that means nothing. The skill is not in wanting a counterfactual. It is in building one that holds up.
But the core idea is worth holding onto, because it is the one that cuts through every dashboard argument. You cannot learn a campaign's true effect by studying only the people who saw it. You need a comparable group who did not. The difference between them is the only honest answer to the question that matters: what would have happened if we had not run this.
Council summary
This post argues that incrementality testing is the only marketing measurement method that answers the question that actually matters: not where credit landed, but whether the ad caused the sale. It makes the counterfactual unmistakable by borrowing the logic of the randomized controlled trial, then sorts the measurement stack by a single test, does the method build a comparison group, and shows attribution and MMM failing it while incrementality passes. The four ways to build that group, randomized holdout, geo test, ghost ads, and synthetic control, are explained in plain language with their real trade-offs, and the post is honest that the method costs reach, needs scale, and answers only one question at a time. The reader's takeaway is practical: treat incrementality as the causal anchor that calibrates the faster methods rather than replaces them, and remember that the value is never in wanting a counterfactual but in building a control group that genuinely holds up.
Comments