A quick introduction to Markov chains and Markov chain Monte Carlo (revised version)

These notes are intended to provide the reader with knowledge of basic concepts of Markov chain Monte Carlo (MCMC) and hopefully also some intuition about how MCMC works. For more thorough accounts of MCMC the reader is referred to e.g. Gilks et al. (1996), Gamerman (1997), or Robert and Casella (1999). Suppose that we are interested in generating samples from a target probability distribution π on R and that π is so complex that we can not use direct methods for simulation. Using Markov chain Monte Carlo methods it is, however, often feasible to generate an ergodic Markov chain X1, X2, . . . which has π as equilibrium distribution, i.e. after a suitable burn-in period m, Xm+1, Xm+2, . . . provides a (correlated) sample from π which can be used e.g. for Monte Carlo computations. Before we turn to MCMC methods we briefly consider in the next section the concepts of Markov chains and equilibrium distributions.