Gibbs sampling is a Markov Chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Since MCMC methods typically suffer from poor scaling when the integral in question is high-dimensional (for example, in problems in Bayesian statistics involving large data sets), researchers have attempted to find ways to speed ...

In this scribe, we are going to review the Parallel Monte Carlo Markov Chain (MCMC) method. First, we will recap of MCMC methods, particularly the Metropolis-Hasting and Gibbs Sampling algorithms. Then we will show the drawbacks of these classical MCMC methods as well as the Naive Parallel Gibbs Sampling approach. Finally, we will come up with the Sequential Monte Carlo and Parallel Inference f...

We give a large family of simple examples where a sharp analysis of the Gibbs sampler can be proved by coupling. These examples involve standard statistical models – exponential families with conjugate priors or location families with natural priors. Many of them seem difficult to succesfully analyze using spectral or Harris recurrence techniques.

A major limitation towards more widespread implementation of Bayesian approaches is that obtaining the posterior distribution often requires the integration of high-dimensional functions. This can be computationally very difficult, but several approaches short of direct integration have been proposed (reviewed by Smith 1991, Evans and Swartz 1995, Tanner 1996). We focus here on Markov Chain Mon...

A rich and  exible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson–Dirichlet process, Ž nite dimensional Dirichlet priors, and beta two-parameter pro...

We deal with the maximum Gibbs ripple of the sampling wavelet series of a discontinuous .function f at a point t ~ R, .for all possible values o.['a satisfying f (t) = ee.f (t 0) + (1 cO.f (t + 0). For the Shannon wavelet series, we make a complete description of all ripples, .for any ot in [0,1]. We show that Meyer sampling series exhibit Gibbs Phenomenon.lor ce < 0 .12495 and ct > 0.306853. W...

Physical systems in nature very often are in thermal equilibrium. Statistical mechanics provides a microscopic theory justifying the relevance of thermal states of matter. However, fully understanding the ubiquity of this class of states from the laws of quantum theory remains an important topic in theoretical physics. The problem can be broken up into two sets of questions: (i) under what cond...

Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an online extension of the collapsed Gibbs sampling (CGS). Our empirical study shows...

In this paper, we investigate combining blocking and collapsing – two widely used strategies for improving the accuracy of Gibbs sampling – in the context of probabilistic graphical models (PGMs). We show that combining them is not straight-forward because collapsing (or eliminating variables) introduces new dependencies in the PGM and in computation-limited settings, this may adversely affect ...

The problem of multilocus linkage analysis is expressed as a graphical model, making explicit a previously implicit connection, and recent developments in the field are described in this context. A novel application of blocked Gibbs sampling for Bayesian networks is developed to generate inheritance matrices from an irreducible Markov chain. This is used as the basis for reconstruction of histo...