We show how the on-line bounded-space bin cover problem can be modeled with a Markov chain. We then use this Markov chain formulation to derive an algorithm for the on-line bounded-space bin cover problem. Our algorithm is designed to perform well in a restrictive environment where it can utilize only very few open bins at each time. We analyze the performance of our algorithm and compare it to...

This paper develops a theoretical framework based on Markov chains for the simple genetic algorithm (operators of reproduction, crossover, and mutation). We prove the existence of a unique stationary distribution for the Markov chain when mutation probability is used as a control parameter. We also show that there is a stationary distribution limit when the control parameter approaches zero. Fi...

a profile hidden markov model (phmm) is widely used in assigning protein sequences to protein families. in this model, the hidden states only depend on the previous hidden state and observations are independent given hidden states. in other words, in the phmm, only the information of the left side of a hidden state is considered. however, it makes sense that considering the information of the b...

We propose an algorithm for the construction of a Hidden Markov Model (HMM) of assigned complexity (number of states of the underlying Markov chain) which best approximates, in Kullback-Leibler divergence rate, a given stationary process. We establish, under mild conditions, the existence of the divergence rate between a stationary process and an HMM, and approximate it with a properly defined ...

We study the slice sampler, a method of constructing a reversible Markov chain with a speciied invariant distribution. Given an independence Metropolis-Hastings algorithm it is always possible to construct a slice sampler that dominates it in the Peskun sense. This means that the resulting Markov chain produces estimates with a smaller asymptotic variance. Furthermore the slice sampler has a sm...

One of the proposed solutions to the equilibrium selection problem for agents learning in repeated games is obtained via the notion of stochastic stability. Learning algorithms are perturbed so that the Markov chain underlying the learning dynamics is necessarily irreducible and yields a unique stable distribution. The stochastically stable distribution is the limit of these stable distribution...

We perform Markov chain Monte Carlo simulations for a Bayesian inference of the GJR-GARCH model which is one of asymmetric GARCH models. The adaptive construction scheme is used for the construction of the proposal density in the Metropolis-Hastings algorithm and the parameters of the proposal density are determined adaptively by using the data sampled by the Markov chain Monte Carlo simulation...

We consider a policy gradient algorithm applied to finite-arm bandit problem with Bernoulli rewards. allow learning rates depend on the current state of rather than using deterministic time-decreasing rate. The forms Markov chain probability simplex. apply Foster–Lyapunov techniques analyze stability this chain. prove that, if are well-chosen, then is transient chain, and converges optimal arm ...

-Markovsemantic indexing algorithm based on the controlled Markov chain modeling framework. Controlled Markov chain models are used to describe the temporal evolution of low-level visual descriptors extracted from the semantic indexing model. Propose a semantic indexing algorithm which uses both text and image retrieval system.The entire user Queries selected by randomly. An image retrieval sys...