Using Bayesian Model Averaging to Calibrate Forecast Ensembles

Ensembles used for probabilistic weather forecasting often exhibit a spread-skill relationship, but they tend to be underdispersive. This paper proposes a principled statistical method for postprocessing ensembles based on Bayesian model averaging (BMA), which is a standard method for combining predictive distributions from different sources. The BMA predictive probability density function (PDF) of any quantity of interest is a weighted average of PDFs centered around the individual (possibly bias-corrected) forecasts, where the weights are equal to posterior probabilities of the models generating the forecasts, and reflect the models’ skill over the training period. The BMA PDF can be represented as an unweighted ensemble of any desired size, by simulating from the BMA predictive distribution. The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members; this can be useful given the cost of running large ensembles. The BMA predictive variance can be decomposed into two components, one corresponding to the between-forecast variability, and the second to the within-forecast variability. Predictive PDFs or intervals based solely on the ensemble spread incorporate the first component but not the second. Thus BMA provides a theoretical explanation of the tendency of ensembles to exhibit a spread-skill relationship but yet to be underdispersive. The method was applied to 48-hour forecasts of sea-level pressure in the Pacific Northwest in January–June, 2000 using the University of Washington MM5 mesoscale ensemble. The predictive PDFs were much better calibrated than the raw ensemble, the BMA forecasts were sharp in that 90% BMA prediction intervals were 62% shorter on average than those produced by sample climatology. As a byproduct, BMA yields a deterministic point forecast, and this had RMSE 11% lower than any of the ensemble members, and 6% lower than the ensemble mean. Similar results were obtained for forecasts of surface temperature.