Incrementality: how to know if your ads actually cause sales

TL;DR

Attribution answers "which touchpoint gets credit." Incrementality answers "would the sale have happened anyway" — a different and more expensive question.
The workhorse tests: brand-search pause, geo holdouts, and platform conversion-lift. Each trades cost for confidence differently.
Incremental ROAS is routinely 40–70% below platform ROAS on retargeting and brand search, and closest to claimed on cold prospecting — test the campaigns where the gap is likely biggest.

The most expensive sentence in performance marketing is "it converts, so it's working." Retargeting your own recent visitors converts beautifully — many of them were coming back anyway. Brand-keyword search ads convert at absurd ROAS — people literally typed your name. Attribution gives those campaigns full credit. Incrementality asks the only question the P&L cares about: what changes if we turn it off?

The three measurement lenses (they answer different questions)

Method	Question	Data needed	Confidence	Cost
Attribution / reconciliation	Which click preceded which order?	UTMs + order join	Directional, always-on	Low
Incrementality tests	Did the spend cause sales?	A controlled on/off split	Causal, per-test	Medium (paused spend or held-out audience)
Media mix modeling	How do channels trade off at the macro level?	2+ years of varied spend	Statistical, quarterly	High

Test 1: the brand-search pause (start here, it's nearly free)

Pause brand keywords in 2–3 cities (or alternate weeks) and watch total branded traffic — paid plus organic — for the same geography. In the classic pattern, organic absorbs 70–95% of the "lost" paid-brand clicks, because people searching your name were finding you regardless. A brand spending ₹80k/month on brand terms that finds 85% absorption just found ₹68k/month of free budget. This is the highest-certainty, lowest-cost test in the playbook.

Test 2: geo holdouts (the D2C workhorse)

Design: pick matched city sets (similar size, trend, seasonality) — say, hold out Pune + Jaipur while Hyderabad + Lucknow keep running.
Run: 3–4 weeks minimum; India's city-level order volumes are noisy and short tests read noise as signal.
Read: compare total store orders (not platform conversions) between test and control geos, versus their pre-test baseline.

Incremental ROASiROAS = (test-geo revenue − expected baseline revenue) ÷ spend in test geos

Worked example: a campaign spends ₹25,000 in the held-out period's mirror geos and platform-reports ₹1,00,000 (4.0×). The geo comparison shows only ₹40,000 of revenue actually disappears when ads stop → iROAS 1.6×. If your break-even MER is 1.7×, this "4× campaign" is marginally unprofitable — a conclusion no dashboard could reach.

Test 3: platform conversion-lift (when spend justifies it)

Meta and Google both run randomised holdout studies (a slice of your target audience never sees the ads). Statistically cleanest, but gated by minimum spend/conversion volume, run on the platform's terms, and still scored against the platform's conversion counting. Use them as a second opinion at ₹15L+/month spend, not as the foundation.

When incrementality testing is the wrong tool

Under ~₹5L/month per channel: the confidence intervals will be wider than the effect you're measuring. Fix reconciliation first — it's free and catches the biggest lies.
During launches, sales or seasonality spikes: baselines break; every result is contaminated.
When you won't act on the answer: a test that can't kill a campaign is theatre. Decide the kill threshold before you run it.

The pragmatic ladder

1) Reconcile claimed vs real revenue (always-on, free). 2) Haircut retargeting and brand-search by their notorious inflation. 3) Brand-pause test to prove the haircut. 4) Geo holdouts on your two biggest campaigns twice a year. 5) MMM only when scale and history justify it. Most brands extract 80% of the value at steps 1–3.

FAQ

How is this different from just watching MER when we cut spend?

That's an uncontrolled version of the same idea — better than nothing, but confounded by seasonality and everything else you changed that week. Geos give you a control group; the discipline is the point.

Which campaigns should we test first?

Where inflation is most likely: retargeting and brand search. Cold prospecting usually tests closest to its claimed numbers — sometimes above (view-through on genuinely new audiences is partly real).

How long and how big must a geo test be?

Rule of thumb: 3–4 weeks, test geos covering 20–30% of spend, and enough weekly orders per geo set (~100+) that a 15–20% effect clears the noise. Below that, extend the duration.

Platform lift studies say our ads are incremental. Trust them?

Directionally useful, but the platform grades its own exam and counts conversions its own way. Cross-check the lift result against store-recorded revenue for the same window before re-allocating budget.

What do we do with an iROAS number once we have it?

Divide claimed by incremental to get that campaign type's deflator, apply it in your weekly allocation until the next test, and re-test twice a year or when the mix shifts materially.