Key Takeway

Are observational methods using good individual-level data “good enough” for ad measurement? Using data from 15 U.S. ad experiments at Facebook, this study showed that current matching and regression-based methods overestimated effectiveness relative to the randomized controlled trials. In half of studies, the estimated percentage increase in purchase outcomes was off by a factor of three across all methods.