Glossary

Incrementality

Incrementality is the principle that only the behavior a program actually caused should be credited to it. It is the discipline behind incremental revenue, and the question every loyalty business case eventually has to answer. Put plainly, it asks how much of everything that happened would have happened anyway.

How it works

Incrementality is established by comparing a group exposed to the program against a comparable group that was not.

The methods, in descending order of how well they hold up:

  • Randomized holdout. Eligible customers are randomly assigned to receive the program or not. Random assignment is what removes self-selection, and it is the only method that produces a measurement rather than an estimate.
  • Geographic or store-level split. The program runs in some regions and not others. This is useful where an individual-level holdout is impractical, which is common in retail.
  • Matched control. Non-members statistically matched to members on prior behavior. This is better than a raw comparison but still vulnerable to differences you did not think to match on.
  • Pre- and post-launch. This is simple and widely used, but anything else happening in the window is attributed to the program, and people often join right when they were about to spend more.

The uncomfortable part is that the better methods all require deliberately withholding something from someone.

Why it matters

Without incrementality, a loyalty program's reported results mostly describe who joined it. Engaged customers join programs; programs then report that their members are engaged.

It changes decisions too. A reward that generates large redemption among customers who would have purchased regardless is a discount, and knowing that is the difference between scaling it and cutting it.

For brands selling through retailers they do not own, incrementality is harder in one way and easier in another. Individual-level holdouts are difficult when the brand cannot see purchases. But geographic splits work well, because retail distribution is already regional.

Good practice is a stated method, an acknowledged margin of error, and a method someone outside marketing agreed to before the test ran.

Agreeing on the method in advance matters more than the sophistication of the method. A simple geographic split designed with finance before launch will carry more weight than an elaborate model produced afterward to explain a number.

Expect incremental figures to be much smaller than headline ones. A smaller figure does not mean the test failed.

Example

A CPG brand runs a points promotion in eight of sixteen regions. Sales rise in all sixteen, seasonally. The gap between the two sets is the incremental effect, and it is roughly a fifth of the total rise. That still justified the promotion, at a fifth of the claimed size.

Common mistake

A common mistake is running the test and then excluding the regions where it did not work. Once the result is filtered, it is no longer a measurement.

Where TrueLoyal® fits: For brands selling through retail, receipt scanning records the store each member purchase came from, which supports the regional comparisons this entry describes when individual holdouts are hard to run. Within the member base, campaigns can also go to one segment and leave a similar one out.

Frequently asked questions

What does incrementality mean in loyalty?

It means crediting a program only with behavior it actually caused, rather than with everything its members did. The test is always what would have happened without the program.

How do you measure incrementality?

Compare an exposed group against a comparable unexposed one. A randomized holdout is strongest, a geographic or store-level split is practical for retail, matched controls are weaker, and pre- and post-launch comparison is weakest because anything else in the window gets attributed to the program.

Why does incrementality matter more than total program results?

Engaged customers join programs, so program results largely describe who joined. Incrementality separates the program's effect from the characteristics of the people it attracted, and that is what decides whether to scale a reward or cut it.

How do brands selling through retail test incrementality?

They usually use geographic or store-level splits rather than individual holdouts, since the brand often cannot observe individual purchases at retail. Retail distribution is already regional, which makes that design practical.

Related terms

Metrics and measurement
Incremental revenue

Incremental revenue is the revenue a loyalty program generated that would not have occurred without it. It is the only revenue figure a program can honestly claim credit for. Total revenue from members is not incremental revenue. Most of it would have happened anyway.

Metrics and measurement
Loyalty program ROI

Loyalty program ROI is the return a program generates against what it costs to run. It is the number that decides whether the program continues at its current budget.

Metrics and measurement
Member vs non-member AOV

Member vs non-member AOV compares the average order value of loyalty program members against customers who are not enrolled, over the same period. It is the comparison most often used to justify a loyalty program, and the one most often misread.

Metrics and measurement
Member vs non-member frequency

Member vs non-member frequency compares how often program members buy against how often non-members buy, over the same period. Of the two standard member-versus-non-member comparisons, this is the more meaningful one. Order value can be moved with a spend threshold in a single campaign. Frequency reflects whether the brand has become a habit.

Close
Schedule a Demo