Ghost Ads: Improving the Economics of Measuring Ad Effectiveness

Garrett A. Johnson, Randall A. Lewis, and Elmar I. Nubbemeyer, 2015, 15-122

To measure the effects of advertising, marketers must know how consumers would behave had they not seen the ads. For example, marketers might compare sales among consumers exposed to a campaign to sales among consumers from a control group who were instead exposed to a public service announcement campaign. While such experimentation is the “gold standard” method, high direct and indirect costs may deter marketers from routinely experimenting to determine the effectiveness of their ad spending.

Featured in Insights from MSI

A better way to measure ad effectiveness

In this report, Garrett Johnson, Randall Lewis, and Elmar Nubbemeyer introduce a methodology they call “ghost ads” which facilitates ad experiments by identifying the control-group counterparts of the exposed consumers in a randomized experiment. With this methodology, consumers in the control group see the mix of competing ads they would see if the experiment’s advertiser had not advertised. To identify the would-be exposed consumers in the control group, ghost ads flag when an experimental ad would have been served to a consumer in the control group.

The authors show that, relative to public service announcement and intent-to-treat A/B tests, “ghost ads” can reduce the cost of experimentation, improve measurement precision tenfold, and work with ad platforms that optimize ad delivery in real time. They also describe a variant “predicted ghost ads” methodology that is compatible with online display advertising platforms. Their implementation of predicted ghost ads on Google’s Display Network records more than 100 million experimental ad impressions daily.

The authors demonstrate their methodology with an online retailer’s display retargeting campaign. Retargeting is a commonplace tactic in which advertisers target users who have considered products on the advertiser’s website. Its effectiveness is controversial because these selected users are likely to purchase even if they were not to see any ads. Using their predicted ghost ads methodology, the authors offer novel evidence that retargeting can work: the retailer’s campaign ads lifted website visits by 17% and purchases by 11%.

The ghost ad methodology promises marketers more accountability for their ad spending. Ghost ad experiments are inexpensive and scalable, which should encourage more advertisers not only to start experimenting, but also to make experimentation routine. Beyond online display ads, this methodology could be applied to search ads and to programmatic TV and radio ads.

Garrett A. Johnson is Assistant Professor of Marketing, Simon Business School, University of Rochester. Randall A. Lewis is Economic Research Scientist, Netflix. Elmar I. Nubbemeyer is Product Manager, Google.

We thank seminar participants at Kellogg, Stanford GSB, UCSD Rady, Abdelhamid Abdou, David Broockman, Hubert Chen, Mitch Lovett, Preston McAfee, John Pau, David Reiley, Robert Saliba, Kathryn Shih, Robert Snedegar, Hal Varian, Ken Wilbur, and many Google employees and advertisers for contributing to the success of this project.  


  • Corporate: FREE
  • Academic: FREE
  • Subscribers: FREE
  • Public: $18.00



Employees of MSI Member Companies enjoy the benefits of complete online access to content, member conferences and networking with the MSI community.



Qualified academics benefit from a relationship with MSI through access to, conferences and research opportunities.



The public is invited to enjoy partial access to content, a free e-newsletter, selected reports and more.




Learn more about becoming a member of the institute

Read More

Stay Informed

The MSI Mailing List

Subscribe to our email list to stay informed about upcoming events, news, etc.