The Metropolis-Hastings algorithm generates correlated samples from a target distribution by constructing a Markov chain which has as its stationary distribution the desired target distribution. One property of this algorithm is that it creates reversible Markov chains. As a result, reversible chains are often used in Monte Carlo simulations. Reversible Markov chains also have the added benefit...

One of the most widely used samplers in practice is the component-wise MetropolisHastings (CMH) sampler that updates in turn the components of a vector spaces Markov chain using accept-reject moves generated from a proposal distribution. When the target distribution of a Markov chain is irregularly shaped, a ‘good’ proposal distribution for one part of the state space might be a ‘poor’ one for ...

Jump Markov linear systems (JMLSs) are linear systems whose parameters evolve with time according to a finite state Markov chain. Given a set of observations, our aim is to estimate the states of the finite state Markov chain and the continuous (in space) states of the linear system. In this paper, we present original deterministic and stochastic iterative algorithms for optimal state estimatio...

Importance sampling, particularly sequential and adaptive importance sampling, have emerged as competitive simulation techniques to Markov–chain Monte–Carlo techniques. We compare importance sampling and the Metropolis algorithm as two ways of changing the output of a Markov chain to get a different stationary distribution.

in this paper, absorbing markov chain models are developed to determine the optimum process mean levels for both a single-stage and a serial two-stage production system in which items are inspected for conformity with their specification limits. when the value of the quality characteristic of an item falls below a lower limit, the item is scrapped. if it falls above an upper limit, the item is ...

The Curveball algorithm is a variation on well-known switch-based Markov Chain Monte Carlo approaches for the uniform sampling of binary matrices with fixed row and column sums. We give a spectral gap comparison between switch chains and the Curveball chain using a decomposition of the switch chain based on Johnson graphs. In particular, this comparison allows us to prove that the Curveball Mar...

‘Iterative conditional fitting’ is a recently proposed algorithm that can be used for maximization of the likelihood function in marginal independence models for categorical data. This paper describes a modification of this algorithm, which allows one to compute maximum likelihood estimates in a class of chain graph models for categorical data. The considered discrete chain graph models are def...

As an important Markov chain Monte Carlo (MCMC) method, the stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. However, S...

The previous results describing the generalization ability of Empirical Risk Minimization (ERM) algorithm are usually based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the first exponential bound on the rate of uniform convergence of the ERM algorithm with Vgeometrically ergodic Markov cha...

We propose a new randomized link scheduling algorithm for wireless networks, called I-CSMA, which is based on a modified version of the Ising model in physics. The main result is that I-CSMA is shown to be throughput-optimal. ICSMA is a generalization of earlier Glauber-dynamics-based, throughput-optimal algorithms such as Q-CSMA in that each earlier algorithm involves a truncated Markov chain ...