Adaptive Independent Metropolis-Hastings by Fast Estimation of Mixtures of Normals

Adaptive Metropolis-Hastings samplers use information obtained from previous draws to tune the proposal distribution. The tuning is carried out automatically, often repeatedly, and continues after the burn-in period. Because the resulting chain is not Markovian, adaptation needs to be done carefully to ensure convergence to the correct ergodic distribution. In this paper we distill recent theoretical advances on nonMarkovian chains into simple guidelines to construct adaptive independent Metropolis-Hastings samplers. We then propose one such sampler in which the flexibility of mixtures of normals is exploited to construct the proposal distribution. To take full advantage of the potential of adaptive samplers it is often desirable to update the mixture of normals frequently and starting early in the chain. Algorithms must therefore be built for speed and reliability. The sampler performance is evaluated with simulated examples and with applications to time-varyingparameter, semi-parametric, and stochastic volatility models.