A/B Testing
Accelerate Experimentation Adoption with Protocols and Property Analysis
Learn more
Marketing Incrementality 101
How do advertisers uncover the true business impact of their spend?
Marketing measurement was always hard, and it’s gotten harder. Over the past few years, privacy changes have shut off classic digital measurement signals. Large digital ad platforms have the market power to report their version of the world and ignore yours. Sophisticated measurement that can navigate this environment was historically limited enterprises with enormous marketing budgets and large data science teams, or advertisers had to lean on the same large digital ad platforms to grade their own homework. Finding trustworthy signal for marketing investments is harder than ever.
Marketing incrementality is the connection of marketing programs to business outcomes – the holy grail in a literally $1 trillion market. Incrementality plays the same role that A/B testing does for digital products, providing a source of truth for what’s moving the needle and not. What makes marketing incrementality tricky is that it encompasses countless advertising and marketing formats, numerous experimental methods, and many point-solution software vendors calling whatever they do “incrementality.”
The basics of marketing incrementality
Incrementality is a marketing program’s effect on your business that would have otherwise not happened without that effort. For example, incrementality would be used to isolate the individual contribution of a branded search campaign or buy-one-get-one-free offer to sales.
Incrementality requires departing from the attribution mindset, where individual actions are tied to clicks or views, and towards a population-level evaluation of what happened after some of the population was treated with a marketing program. In this regard, it’s similar to clinical trials used in medicine, where some patients may receive an experimental medicine and some may receive a placebo, but at the end, both sets of patients are evaluated on the same type of health metrics.
Common marketing incrementality methods
Two broadly popular methods for evaluating marketing incrementality are:
- A/B tests: Very similar to the clinical trial example mentioned above, A/B tests are frequently used when marketers have access to user-level data, like for web content, email, push notifications, or text messaging.
- Geo-lift tests: Advertising programs usually don’t make user-level data available, so its effects must be evaluated at higher geographic levels like regions, states, or countries. With this, similarities between regions are exploited to isolate the effects of the marketing program under question.
Both of these methods can be employed to measure incrementality across everything from advertising to web content to SEO to in-store promotions to CRM/lifecycle campaigns. A/B testing is a well-trodden area but with many best practices that are key to implement. Given the key position that geo-testing has for marketing incrementality testing, we’ll spend more time on that in this post. For more about A/B/n testing, read this article. “What is A/B/n Testing? Easy Guide with Examples”
Geo-testing: the basics
Let’s say you ran a billboard campaign in Chicago and want to know if it increased your sales. An ideal experiment test would be to compare Chicago to a perfectly parallel universe Chicago where the only difference was the lack of your billboard campaign. You obviously can’t do that, so we instead create a synthetic version of Chicago using the data from other cities that behave similarly where the ad didn’t run. You’d then compare sales in your real Chicago to the “synthetic Chicago to see the billboard campaign’s effect. It won’t be exactly perfect, but it’ll be a pretty good comparison useful to make business decisions.
Advertisers have long used a technique called “market matching” where a test region is matched with a single control region and differences are compared – the synthetic control methodology is similar in initial spirit but then can use hundreds of regions to build the most accurate control representation of the market being tested.
Benefits of Geo-testing:
- Provides the most accurate market-level impact assessment
- Geotesting’s universality and ground-truth characteristics help calibrate attribution and Media Mix Modeling (MMM) results
- Doesn't rely on user-level data, making it privacy-compliant
- Offers an alternative to platform-specific "lift studies.” These features are powerful but rely on black-boxed data and metrics that make it difficult to compare to your real business results.
- Effective for measuring top-of-funnel channels like YouTube and out-of-home advertising, or interaction-effect heavy channels like branded search ads.
Challenges of Geo-testing
While powerful, geo-testing comes with its own set of challenges:
- Requires complex statistical analyses and modeling traditionally available to only large digitally-native companies with significant data science expertise.
- Demands consistent, accurate data collection across regions that are the same as your business metrics
- Events like weather or local market disruptions can skew results, and experienced practitioners take care in completely separating test and control groups
- May need longer test periods or larger effect sizes compared to user-level A/B tests
The Future of Incrementality Testing
While there's been an exciting wave of investment and interest in tooling for marketing incrementality over the past year or two, we can still expect plenty of exciting developments to come. There's plenty more to do to build upon our existing tools for modeling treatment intensity, meta-analysis for decision-making, and making it easier to choose the right geos for your test. Integrations with the broader marketing and data stack will also become a key part of driving maturity for organizations.
Unified incrementality testing represents a significant leap forward in marketing measurement and optimization. It enables organizations to:
- Make data-driven decisions with confidence
- Allocate marketing budgets more effectively
- Demonstrate the true value of marketing initiatives to stakeholders
This ability to accurately measure marketing incrementality will become a crucial competitive advantage as the broader ad ecosystem becomes increasingly complex and privacy-conscious. Organizations that embrace this approach will be better positioned to navigate modern marketing challenges and drive sustainable growth.
Now is the perfect time to start building your incrementality testing muscle -implement these tools across your organization to unlock the full potential of your marketing efforts. Whether you're just starting or looking to enhance existing capabilities, investing in unified incrementality testing is a strategic move that will pay dividends in the years to come.
For an easy way to get started, take a look at Eppo Geolift. Not an Eppo customer yet? Request a demo and we'd be happy to show you around!
Table of contents
Ready to go from knowledge to action?
Talk to our team of experts and see why companies like Twitch, DraftKings, and Perplexity use Eppo to power experimentation for every team.
Get a demo
.webp)
