Let X be a Markov additive process with continuous paths and a finite background Markov process J so that X evolves as Brownian motion with drift r(i) and variance σ2(i) when J(t) = i. Assuming that J is eventually absorbed at some state a, the density f(x) of Z = X(ζ) is found where ζ is the absorbtion time. The form of f(x) is non-smooth at x = 0, but the distributions of Z+ and Z− are both o...

A central limit theorem is established for additive functions of a Markov chain that can be constructed as an iterated random function. The result goes beyond earlier work by relaxing the continuity conditions imposed on the additive function, and by relaxing moment conditions related to the random function. It is illustrated by an application to a Markov chain related to fractals.

In the paper, the law of the iterated logarithm for additive functionals of Markov chains is obtained under some weak conditions, which are weaker than the conditions of invariance principle of additive functionals of Markov chains in M. Maxwell and M. Woodroofe [7] (2000). The main technique is the martingale argument and the theory of fractional coboundaries.

In the paper, the law of the iterated logarithm for additive functionals of Markov chains is obtained under some weak conditions, which are weaker than the conditions of invariance principle of additive functionals of Markov chains in M. Maxwell and M. Woodroofe [7] (2000). The main technique is the martingale argument and the theory of fractional coboundaries.

The notion of N-ple Markov process is defined in a quite general framework and it is shown that N-ple Markov processes-arel inear combinationso f some martingales

Production and non-productive equipment and personnel delays are a critical element of any production system. The frequency and length of delays impact heavily on the production and economic efficiency of these systems. Machining processes in wood industry are particularly vulnerable to productive and non-productive delays. Whereas, traditional manufacturing industries usually operate on homoge...

We prove that the large deviation principle holds for a class of processes inspired by semi-Markov additive processes. For the processes we consider, the sojourn times in the phase process need not be independent and identically distributed. Moreover the state selection process need not be independent of the sojourn times. We assume that the phase process takes values in a finite set and that t...

We derive novel upper and lower finite length bounds of the error probability in joint source-channel coding when the source is distributed according to an ergodic Markov process and the channel is a Markovian additive channel or a Markovian conditional additive channel. We show that these bounds are tight in the moderate deviations regime and for a certain interval of rates, also in the large ...

Specifying the reward function of a Markov decision process (MDP) can be demanding, requiring human assessment of the precise quality of, and tradeoffs among, various states and actions. However, reward functions often possess considerable structure which can be leveraged to streamline their specification. We develop new, decisiontheoretically sound heuristics for eliciting rewards for factored...