attribution · measurement-paradigms

Measurement Paradigms — MTA vs. MMM vs. Incrementality

Attribution models (see attribution-models.md) split credit within your tracked data. They can't tell you what would have happened anyway. That's what these three paradigms are for — increasingly rigorous, increasingly expensive ways to get closer to causality. Use this reference to help a user pick, and to explain how a test reads (not how to run the statistics).

The three, compared

	MTA (multi-touch)	MMM (media mix modeling)	Incrementality (experiments)
Approach	Bottom-up: stitch user-level touches, apply a model	Top-down: regress outcomes vs. spend/factors over time	Controlled: withhold exposure from a group, measure the difference
Answers	"Which touchpoints are on converting journeys?"	"What's each channel's aggregate contribution?"	"Did this channel cause lift I wouldn't have gotten?"
Granularity	Per user, per touch	Per channel, per week	Per test (one channel/tactic at a time)
Data needed	Clean cross-device user-level tracking	2–3 yrs weekly data + spend variation	Ability to withhold + enough volume for significance
Reacts	Real-time	Slowly (weeks/quarters)	Per test cycle
Handles offline/brand	No	Yes	Yes (if you can split exposure)
Privacy-durable	Weak (cookie/ID loss)	Strong (aggregate)	Strong (aggregate)
Cost/effort	Low–medium	High	Medium–high

MTA — multi-touch attribution

What it is: the user-level approach most marketers mean by "attribution" — join a person's touches, apply first/linear/position/data-driven credit.

Where it shines: day-to-day, tactical decisions. "Is this campaign showing up on converting journeys?" Fast, granular, cheap if your tracking exists.

Why it's structurally weakening: MTA depends on tracking one person across touches and devices, and that data keeps eroding — Safari/Firefox ITP-style cookie limits, iOS ATT, browser consent gating, ad blockers, and ordinary cross-device behavior (research on mobile, buy on desktop). (Chrome's third-party-cookie deprecation was announced, then walked back, so "cookies are going away" is no longer the clean story — but everything else on that list already limits user-level tracking today.) MTA doesn't announce the gap: it silently under-measures anything it can't follow (dumping it into direct) and over-measures what it can see. So treat MTA as incomplete and directionally biased — not a clean lower bound (its retargeting/branded numbers are often over-stated) — and never the sole basis for a big reallocation.

Use it for: ongoing optimization and trend-watching — never as the sole basis for a big budget reallocation.

MMM — media/marketing mix modeling

What it is: a top-down statistical model (historically regression; modern open-source options like Meta's Robyn or Google's Meridian) that explains an outcome (revenue, signups) as a function of spend per channel plus controls (seasonality, promotions, price, macro). It never looks at individuals — it's all aggregate time-series, which is exactly why privacy changes don't touch it.

What it uniquely gives you:

Offline + brand + hard-to-track channels — TV, podcasts, OOH, PR, organic — because it works on aggregate spend and outcomes, not clicks.
Diminishing returns / saturation curves — where the next dollar in a channel stops paying off.
A portfolio view — how the whole mix drives the outcome, not credit for one journey.

What it costs and where it's weak:

Needs 2–3 years of weekly data and real variation in spend — if you always spend the same on Meta, the model can't learn Meta's effect. You sometimes have to deliberately vary budgets to feed it.
Correlational and slow — it sees what moved together historically; it reacts in quarters, not days. It can't tell you what to do with tomorrow's campaign.
Sensitive to specification — garbage controls, garbage coefficients. It's a real modeling exercise, not a dashboard toggle.

Use it for: annual/quarterly budget allocation across a material, multi-channel (esp. offline-inclusive) spend. Validate its coefficients with incrementality tests — MMM says "Meta contributed X"; a holdout proves whether that's causal.

Incrementality — the experiments

What it is: the only paradigm that measures causality directly. Split your audience into an exposed group and a withheld/control group; the difference in outcomes is the incremental lift — conversions you got because of the channel, not ones that would have happened anyway.

Common designs:

Geo holdout / geo-lift — run the channel in some regions, hold it out of comparable ones; compare outcomes. The workhorse for channels you can't split at the user level. (Meta's GeoLift, Google's geo experiments.)
PSA tests — the control group is shown a public-service ad in your ad's place, so both groups are equally targeted and "ad-exposed"; the only difference is whether they saw your ad. Isolates ad effect from audience-quality bias (the exposed group isn't just "people the algorithm judged likely to convert").
Ghost ads — the control group is held out of the auction but the platform logs the ad that would have served them (no placeholder is shown); you compare converters among the would-have-been-exposed vs. actually-exposed. Cleaner and cheaper than PSAs (no wasted PSA spend), and the modern default where the platform supports it.
Intent-to-treat / on-off (pulse) tests — turn a channel fully off for a defined window, watch what happens to total conversions. Crude but revealing, especially for "is branded search paid cannibalizing organic?"
Holdout audiences — withhold a random % from a retargeting or email program; the delta is the program's true lift.

How to read a test (not run the stats)

You don't need to compute significance by hand, but you must read a result honestly:

Lift = exposed rate − control rate. If exposed geos converted at 4.2% and control at 3.6%, incremental lift is ~0.6pp — the rest of that 4.2% would have converted anyway. This is why last-click ROAS is almost always overstated: it counts the whole 4.2%.
Check the confidence interval / significance. "5% lift, but the interval spans −2% to +12%" means you learned nothing — the test was underpowered. Insist on enough volume/duration before believing a point estimate.
Watch for contamination. Control users who were reached anyway (cross-device, spillover between geos) shrink the measured gap. A "no lift" result can be a leaky test, not a dead channel.
Translate to a decision. Incremental CPA = spend ÷ incremental conversions (not total). This is the number that should drive budget — and it's usually worse than the platform's reported CPA, which is the point.

Use it for: the highest-stakes questions and the tiebreakers — "does retargeting actually do anything?", "is branded-search paid just buying clicks we'd get free?", "which of our two biggest channels is really driving growth?" You can only test a few things at a time, so spend those tests on the decisions that matter most.

Putting them together (the mature stack)

They're layers, not competitors:

MTA for daily/weekly tactical optimization and trend-watching — cheap, granular, directional.
MMM for quarterly/annual portfolio allocation across the full mix including offline — durable, holistic.
Incrementality as the calibration and tiebreaker — the ground-truth that keeps MTA and MMM honest, run on your biggest bets.

Scale to the user: most SMBs need good UTMs + last-non-direct + a self-reported survey + the occasional on/off test — not an MMM. Bring MMM in when offline/brand spend is material and MTA visibly can't see it. Bring in formal incrementality when a single channel's budget is big enough that being wrong about it is expensive. Match the rigor to the size of the decision.