A transformation mapping a labelled Markov chain to a simple stochastic game is presented. In the resulting simple stochastic game, each vertex corresponds to a pair of states of the labelled Markov chain. The value of a vertex of the simple stochastic game is shown to be equal to the probabilistic bisimilarity distance, a notion due to Desharnais, Gupta, Jagadeesan and Panangaden, of the corre...

the problem we investigate deals with the optimal assignment of resources to the activities of a stochastic project network. we seek to minimize the expected cost of the project include sum of resource utilization costs and lateness costs. we assume that the work content required by the activities follows an exponential distribution. the decision variables of the model are the allocated resourc...

Abstract. I describe a new Markov chain method for sampling from the distribution of the state sequences in a non-linear state space model, given the observation sequence. This method updates all states in the sequence simultaneously using an embedded Hidden Markov model (HMM). An update begins with the creation of a “pool” of K states at each time, by applying some Markov chain update to the c...