What is incrementality — and why your dashboard can’t show it
Incrementality is the share of conversions your marketing actually caused — not the ones it merely sat next to. It’s the difference between what happened and what would have happened anyway. Most dashboards can’t tell those two apart, which is why a campaign can look like a winner while adding nothing to the bottom line. Here’s how to think about it, and what it takes to actually measure it.
By William Walczak — CEO, Hiilite Creative Group Inc.The one-sentence definition
Incrementality is the share of conversions that happened because of your marketing — conversions that would not have occurred otherwise. Everything else is a conversion that was coming anyway: the customer who already typed your name into Google, the repeat buyer who was due for a reorder, the loyal regular who would have walked in regardless of whether they saw your ad.
This matters because spending money to reach people who were already going to buy is not marketing — it’s a rebate you’re paying to yourself. The whole job of a campaign is to create purchases that weren’t going to happen. That created portion is the increment. It’s the only part worth paying for, and it’s the part your reporting tools are worst at showing you.
The counterfactual: what would have happened anyway
To know what your marketing caused, you need a baseline — a picture of the world where you didn’t run the campaign. That imaginary world is called the counterfactual, and it’s the heart of every honest measurement question. Sales with the campaign, minus sales in the counterfactual, equals incremental lift.
The catch is that the counterfactual never actually happens. You ran the campaign, so you only ever observe one version of reality. You can’t rewind the month and skip the ads to compare. This is why measurement is genuinely hard — not because the math is complicated, but because the comparison you need is invisible by default. The art is constructing a credible stand-in for that missing world. We unpack that idea on its own in the counterfactual.
Why a dashboard shows totals, not lift
Open any ad platform or analytics dashboard and you’ll see totals: conversions, revenue, ROAS, cost per acquisition. These are real numbers, but they answer the wrong question. They tell you how many conversions were associated with your marketing — not how many it caused. The platform happily counts a sale as a win even if that customer was already heading to checkout.
Last-click attribution makes this worse. It hands full credit to whatever the customer touched last — usually a branded search or a retargeting ad shown to someone already deep in the funnel. Those touchpoints look spectacular on the dashboard precisely because they sit closest to conversions that were going to happen anyway. The tool isn’t lying; it’s just measuring correlation and calling it credit.
So a campaign can post a 6× ROAS and still be worthless — if those buyers would have converted without ever seeing the ad, the true incremental return is zero. The dashboard can’t flag this, because it has no counterfactual. It only ever sees the world where the ad ran. To see the difference between attribution, modelling, and true lift, compare MMM vs incrementality vs attribution.
How you actually measure it: holdouts and geo-tests
If the counterfactual won’t reveal itself, you have to build one. The cleanest way is a holdout: deliberately withhold your marketing from a randomly chosen group of customers (or regions, or audiences) while showing it to everyone else. Because the two groups are otherwise alike, any gap in their conversion rates is the lift your marketing produced. That gap is incrementality, measured directly.
When you can’t split individuals — common for brand campaigns, local businesses, or offline media — you can run a geo-test instead: turn the campaign on in some cities or regions and off in comparable ones, then compare the difference. Same logic, larger unit. Either way, the held-back group is your living counterfactual: the version of the world where you didn’t spend, observed for real instead of guessed.
- Holdout test — randomly suppress ads for a slice of your audience; the converted-rate gap vs the rest is your lift.
- Geo-test — run the campaign in some regions, hold it back in matched ones, and compare; good for brand and offline media.
- A/B / experiment — the same controlled-comparison idea applied to a single change, like a new offer or landing page.
The SMB takeaway: the gap vs do-nothing
You don’t need a data-science team to act on this. You need one habit: before you trust a marketing number, ask compared to what? If the answer is “compared to not running it,” you’re looking at incrementality. If the answer is “compared to nothing — it’s just the total,” treat it as a vanity figure until proven otherwise.
Start small. Hold back one campaign from one region or audience for a few weeks and watch whether your results actually fall. If sales hold steady without the spend, that budget was buying conversions you already had. If they drop, you’ve found a real engine — and now you know exactly what it’s worth. That single discipline reshapes how you allocate every dollar, which is the foundation of our Growth Mapping framework.
The bottom line: the only number that proves marketing worked is the gap between what happened and what would have happened if you’d done nothing. Everything else is decoration.
Keep reading
Questions, answered
Isn’t a high ROAS proof that my ads are working?
Not on its own. ROAS counts revenue associated with your ads, including from customers who would have bought anyway. A campaign can show a strong ROAS and still produce zero incremental sales if it’s mostly reaching people already heading to checkout. The proof is whether sales actually drop when you turn the campaign off — that gap is the real return.
Why can’t my analytics tool just calculate incrementality for me?
Because incrementality requires a counterfactual — the version of the world where you didn’t run the campaign — and your analytics tool only ever observes the world where you did. It can count and attribute what happened, but it has nothing to compare against. Measuring lift takes a deliberate comparison, like a holdout or a geo-test, that you have to set up on purpose.
I’m a small business. Is incrementality testing realistic for me?
Yes, in a scaled-down form. You don’t need perfect statistics to hold one campaign back from one region or audience for a few weeks and see whether results fall. Even a rough geo-test tells you more than any dashboard total, because it answers the only question that matters: did this spend create sales that weren’t coming anyway?
More from this series
Seven plain-English pieces on measuring marketing and turning tests into durable capability:
Want to know whether your marketing is the reason?
One conversation to map what to measure and how we’d prove it.