The paper studies a probabilistic notion of causes in Markov chains that relies on the counterfactuality principle and probability-raising property. This is motivated by use for monitoring purposes where aim to detect faulty or undesired behaviours before they actually occur. A cause set finite executions system after which probability effect exceeds given threshold. We introduce multiple types...

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...

Many approaches to tackle the state explosion problem of Markov chains are based on the notion of lumpability, which allows computation of measures using the quotient Markov chain, which, in some cases, has much smaller state space than the original one. We present, for the first time, a symbolic algorithm and its implementation for the lumping of Markov chains that are represented using Multi-...

The Metropolis-Hastings algorithm generates correlated samples from a target distribution by constructing a Markov chain which has as its stationary distribution the desired target distribution. One property of this algorithm is that it creates reversible Markov chains. As a result, reversible chains are often used in Monte Carlo simulations. Reversible Markov chains also have the added benefit...

We discuss recent work on the algorithmic analysis of systems involving recursion and probability. Recursive Markov chains extend ordinary finite state Markov chains with the ability to invoke other Markov chains in a potentially recursive manner. They offer a natural abstract model of probabilistic programs with procedures, and generalize other classical well-studied stochastic models, eg. Mul...