Data from 15 U.S. ad experiments at Facebook demonstrate that current matching and regression-based methods using observational data overestimate effectiveness relative to randomized controlled trials. In half of studies, the estimated percentage increase in purchase outcomes was off by a factor of three across all methods.