Combined Forecasts: What to Do When One Model Isn’t Good Enough

SAS® High-Performance Forecasting 4.1 offers a new, innovative process for automatically combining forecasts. Forecast combination, also called ensemble forecasting, is the subject of many academic papers in statistical and forecasting journals; it is a known technique for improving forecast accuracy and reducing variability of the resulting forecasts. By integrating these methods into a single software system, SAS High-Performance Forecasting 4.1 surpasses the functionality of any existing software system that incorporates this capability. This paper describes this new capability and includes examples that demonstrate the use and benefits of this new forecast combination process. INTRODUCTION The M-Competition is a periodic public competition of forecasting methods. This competition pits researchers and their methodologies against one another to forecast a collection of time series data. The goal of M-Competition is to evaluate the skills of researchers by the accuracy of their predictions. Makridakis and Hibon (2000) wrote the following as one of their four conclusions about the results of a recent M3 competition: “The accuracy of the combination of various methods outperforms, on average, the specific methods being combined and does well in comparison with other methods.” The lesson from this statement is that a combination of forecasts from simple models can add substantial value in terms of enhancing the quality of the forecasts produced, but the statement also admits that combinations might not always perform better than a suitably crafted model. With SAS High-Performance Forecasting 4.1, these combinations can be generated automatically and integrated into a well-proven framework for model selection and forecasting. The value of the tool is enhanced, and overall forecast quality can be improved by the use of combined forecasts. The remaining sections of this paper discuss several aspects of forecast combination and provide examples that demonstrate the use of combined forecasts in the SAS High-Performance Forecasting procedures. First, there is a short casestudy to explore why forecast combinations can be beneficial. That is followed by a brief summary of the mathematical details of forecast combination, a discussion of the forecast combination process flow, and a discussion of the model selection process as extended to incorporate combined forecasts. Several examples follow that demonstrate different aspects of combined forecast usage. These include SAS statements and samples of the output they produce. For readers who are not familiar with the SAS High Performance Forecasting product, see the SAS High-Performance Forecasting 4.1: User’s Guide for information about the various SAS procedures used in the examples. It is beyond the scope of this paper to describe these procedures and their use except as they pertain to the new features related to forecast combinations. WHY COMBINE FORECASTS? When might the combined method of forecasting outperform other forecasting methods? The time series for monthly oil production for the Central region in the Gulf of Mexico (2006) makes a compelling case for combined forecasts. The oil production data span the period from 1996 to 2006. PROC HPFDIAGNOSE is run to generate an autoregressive integrated moving average (ARIMA) model and an exponential smoothing model (ESM) for this series, and then the HPFENGINE procedure is run to select the best model based on a holdout sample of six months. Figure 1 shows the results of the PROC HPFENGINE run. Although both of these models might be sufficiently accurate expressions of the data-generating process, would some weighted average of the forecasts produce a better-quality forecast? 1 Statistics and Data Analysis SAS Global Forum 2012