Bounded truncation error for long-run averages in infinite Markov chains
نویسندگان
چکیده
منابع مشابه
Bounded truncation error for long-run averages in infinite Markov chains
We consider long run averages of additive functionals on infinite discrete-state Markov chains, either continuous or discrete in time. Special cases include long run average costs or rewards, stationary moments of the components of ergodic multi-dimensional Markov chains, queueing network performance measures, and many others. By exploiting Foster-Lyapunov-type criteria involving drift conditio...
متن کاملInfinite Dimensional Entangled Markov Chains
We continue the analysis of nontrivial examples of quantum Markov processes. This is done by applying the construction of entangled Markov chains obtained from classical Markov chains with infinite state–space. The formula giving the joint correlations arises from the corresponding classical formula by replacing the usual matrix multiplication by the Schur multiplication. In this way, we provid...
متن کاملSliding Window Abstraction for Infinite Markov Chains
We present an on-the-fly abstraction technique for infinite-state continuous-time Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic modeling of biological systems. We approximate the transient probability distributions at various time inst...
متن کاملLong-Run Rewards for Markov Automata
Markov automata are a powerful formalism for modelling systems which exhibit nondeterminism, probabilistic choices and continuous stochastic timing. We consider the computation of long-run average rewards, the most classical problem in continuous-time Markov model analysis. We propose an algorithm based on value iteration. It improves the state of the art by orders of magnitude. The contributio...
متن کاملModel Checking Infinite-State Markov Chains
In this paper algorithms for model checking CSL (continuous stochastic logic) against infinite-state continuous-time Markov chains of so-called quasi birth-death type are developed. In doing so we extend the applicability of CSL model checking beyond the recently proposed case for finite-state continuous-time Markov chains, to an important class of infinite-state Markov chains. We present synta...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Applied Probability
سال: 2015
ISSN: 0021-9002,1475-6072
DOI: 10.1239/jap/1445543835