Testing theories with learnable and predictive representations

We study the problem of testing an expert whose theory has a learnable and predictive parametric representation, as do all standard processes used in Bayesian statistics. We design a test in which the expert is required to submit a date T by which he will have learned enough to deliver sharp predictions about future frequencies. His forecasts are then tested according to a simple hypothesis test. We show that this test passes an expert who knows the data-generating process and cannot be manipulated by an uninformed one. Such a test is not possible if the theory is unrestricted. ∗ We are grateful to Eddie Dekel, Lance Fortnow, Ehud Kalai, Wojciech Olszewski, Marcin Peski, Phil Reny, Rakesh Vohra, and seminar audiences at Northwestern, Montreal, Technion, Minnesota, Princeton, and Maryland. We also thank Nenad Kos, Yang Zhang and Jie Gong for their comments and careful proofreading. † Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston IL 60208. e-mail:[email protected]. Research page : http://www.kellogg.northwestern.edu/faculty/alnajjar/htm/index.htm ‡ Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston IL 60208, and Department of Economics, University of Pennsylvania. e-mail: . § Davidson Faculty of Industrial Engineering and Management,Technion, Haifa 32000, Israel. e-mail: . ¶ Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston IL 60208. e-mail: [email protected] Research page : http://www.kellogg.northwestern.edu/faculty/weinstein/htm/index.htm