Censoring Markov Chains and Stochastic Bounds
نویسندگان
چکیده
We show how to combine censoring technique for Markov chain and strong stochastic comparison to obtain bounds on rewards and the first passage time. We present the main ideas of the method, the algorithms and their proofs. We obtain a substantial reduction of the state space due to the censoring technique. We also present some numerical results to illustrate the effectiveness of the method.
منابع مشابه
Stochastic bounds for a single server queue with general retrial times
We propose to use a mathematical method based on stochastic comparisons of Markov chains in order to derive performance indice bounds. The main goal of this paper is to investigate various monotonicity properties of a single server retrial queue with first-come-first-served (FCFS) orbit and general retrial times using the stochastic ordering techniques.
متن کاملModel Checking by Censoring Markov Chains and Stochastic Comparison
We study how to combine the censoring technique for Markov chains and the strong stochastic comparison to perform model checking of discrete-time Markov chains. Our goal is to reduce the complexity of the model checking in order to be able to consider numerically intractable models. In model checking we do not need the exact values but we must decide if the required guarantees are satisfied or ...
متن کاملStochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry
We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...
متن کاملSome Specialized Comparison Techniques for Doubly Stochastic Finite Markov Chains
This paper considers certain comparison techniques involving Markov chains with transition matrices Pa = aI+(1?a)P where P is the transition matrix of a doubly stochastic Markov chain. This paper provides upper bounds for how far the Markov chain with transition matrix Pb is from uniformly distributed after n steps. These upper bounds involve how far the Markov chain with transition matrix Pa i...
متن کاملStochastic Bounds for Partially Generated Markov Chains: An Algebraic Approach
We propose several algorithms to obtain bounds based on Censored Markov Chains to analyze partially generated discrete time Markov chains. The main idea is to avoid the generation of a huge (or even infinite) state space and to truncate the state space during the visit. The approach is purely algebraic and provides element-wise and stochastic bounds for the CMC.
متن کامل