Judgmental Bootstrapping: Inferring Experts' Rules for Forecasting

Judgmental bootstrapping is a type of expert system. It translates an experts' rules into a quantitative model by regressing the experts' forecasts against the information that he used. Bootstrapping models apply an experts' rules consistently, and many studies have shown that decisions and predictions from bootstrapping models are similar to those from the experts. Three studies showed that bootstrapping improved the quality of production decisions in companies. To date, research on forecasting with judgmental bootstrapping has been restricted primarily to cross-sectional data, not time-series data. Studies from psychology, education, personnel, marketing, and finance, showed that bootstrapping forecasts were more accurate than forecasts made by experts using unaided judgment. They were more accurate for eight of eleven comparisons, less accurate in one, and there were two ties. The gains in accuracy were generally substantial. Bootstrapping can be useful when historical data on the variable to be forecast are lacking or of poor quality; otherwise, econometric models should be used. Bootstrapping is most appropriate for complex situations, where judgments are unreliable, and where experts' judgments have some validity. When many forecasts are needed, bootstrapping is cost-effective. If experts differ greatly in expertise, bootstrapping can allow one to draw upon the forecasts made by the best experts. Bootstrapping aids learning; it can help to identify biases in the way experts make predictions, and it can reveal how the best experts make predictions. Finally, judgmental bootstrapping offers the possibility of conducting ?experiments@ when the historical data for causal variables have not varied over time. Thus, it can serve as a supplement for econometric models. Disciplines Business | Marketing Comments Suggested Citation: Armstrong, J.S. Judgmental Bootstrapping: Inferring Experts' Rules for Forecasting. In Principles of Forecasting: A Handbook for Researchers and Practitioners (Ed. J. Scott Armstrong). Kluwer, 2001. Publisher URL: http://www.springer.com/business+%26+management/business+for+professionals/book/ 978-0-7923-7930-0 This book chapter is available at ScholarlyCommons: http://repository.upenn.edu/marketing_papers/150 Principles of Forecasting: A Handbook for Researchers and Practitioners, J. Scott Armstrong (ed.): Norwell, MA: Kluwer Academic Publishers, 2001. Judgmental Bootstrapping: Inferring Experts= Rules for Forecasting J. Scott Armstrong The Wharton School University of Pennsylvania ABSTRACT Judgmental bootstrapping is a type of expert system. It translates an expert=s rules into a quantitative model by regressing the expert=s forecasts against the information that he used. Bootstrapping models apply an expert=s rules consistently, and many studies have shown that decisions and predictions from bootstrapping models are similar to those from the experts. Three studies showed that bootstrapping improved the quality of production decisions in companies. To date, research on forecasting with judgmental bootstrapping has been restricted primarily to cross-sectional data, not time-series data. Studies from psychology, education, personnel, marketing, and finance, showed that bootstrapping forecasts were more accurate than forecasts made by experts using unaided judgment. They were more accurate for eight of eleven comparisons, less accurate in one, and there were two ties. The gains in accuracy were generally substantial. Bootstrapping can be useful when historical data on the variable to be forecast are lacking or of poor quality; otherwise, econometric models should be used. Bootstrapping is most appropriate for complex situations, where judgments are unreliable, and where experts= judgments have some validity. When many forecasts are needed, bootstrapping is cost-effective. If experts differ greatly in expertise, bootstrapping can allow one to draw upon the forecasts made by the best experts. Bootstrapping aids learning; it can help to identify biases in the way experts make predictions, and it can reveal how the best experts make predictions. Finally, judgmental bootstrapping offers the possibility of conducting ?experiments@ when the historical data for causal variables have not varied over time. Thus, it can serve as a supplement for econometric models.Judgmental bootstrapping is a type of expert system. It translates an expert=s rules into a quantitative model by regressing the expert=s forecasts against the information that he used. Bootstrapping models apply an expert=s rules consistently, and many studies have shown that decisions and predictions from bootstrapping models are similar to those from the experts. Three studies showed that bootstrapping improved the quality of production decisions in companies. To date, research on forecasting with judgmental bootstrapping has been restricted primarily to cross-sectional data, not time-series data. Studies from psychology, education, personnel, marketing, and finance, showed that bootstrapping forecasts were more accurate than forecasts made by experts using unaided judgment. They were more accurate for eight of eleven comparisons, less accurate in one, and there were two ties. The gains in accuracy were generally substantial. Bootstrapping can be useful when historical data on the variable to be forecast are lacking or of poor quality; otherwise, econometric models should be used. Bootstrapping is most appropriate for complex situations, where judgments are unreliable, and where experts= judgments have some validity. When many forecasts are needed, bootstrapping is cost-effective. If experts differ greatly in expertise, bootstrapping can allow one to draw upon the forecasts made by the best experts. Bootstrapping aids learning; it can help to identify biases in the way experts make predictions, and it can reveal how the best experts make predictions. Finally, judgmental bootstrapping offers the possibility of conducting ?experiments@ when the historical data for causal variables have not varied over time. Thus, it can serve as a supplement for econometric models.