Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions

Google and Bing employ generalized second price (GSP) auctions to allocate billions of dollars of sponsored search advertising. The theoretical work establishing the VCG equivalence of the GSP hinges on strong, and largely untested, assumptions about the causal impact of ad position on user click probability and the value of resulting clicks. We re-purpose internal business experimentation to test these assumptions using a broad cross-section of advertisers. We find a minimal, homogenous impact of position on average click quality, providing a valuable assurance in that domain. In contrast, we find substantial heterogeneity of the impact on click through rates. For brand queries, off-brand competitors show much steeper click curves than the advertiser matching the query. For generic queries, higher quality and less popular websites benefit more from position. These findings strongly reject the conventional multiplicatively-separable model and the associated VCG equivalence, raising serious question about the efficiency properties of the GSP in sponsored search. ∗We would like to thank the Microsoft Corporation for allowing us to publish this work and Jim Andreoni, Susan Athey, Dean Foster, Sebastién Lahaie, R. Preston McAfee, Andrey Simonov and Vasilis Syrgkanis for helpful discussion and comments. †Much of this work was done while Goldman was an intern at Microsoft Research.