Birth, Death and Conditioning of Markov Chains

Lecture 3: Continuous times Markov Chains. Poisson Process. Birth and Death Process. 1 Continuous Time Markov Chains

In this lecture we will discuss Markov Chains in continuous time. Continuous time Markov Chains are used to represent population growth, epidemics, queueing models, reliability of mechanical systems, etc. In Continuous time Markov Process, the time is perturbed by exponentially distributed holding times in each state while the succession of states visited still follows a discrete time Markov ch...

Spectral Computations for Birth and Death Chains

We consider the spectrum of birth and death chains on a n-path. An iterative scheme is proposed to compute any eigenvalue with exponential convergence rate independent of n. This allows one to determine the whole spectrum in order n elementary operations. Using the same idea, we also provide a lower bound on the spectral gap, which is of the correct order on some classes of examples.

Spatial Birth-and-Death Markov Processes

Two-Time-Scale Markov Chains and Applications to Quasi-Birth-Death Queues

Aiming at reduction of complexity, this work is concerned with two-time-scale Markov chains and applications to quasi-birth-death queues. Asymptotic expansions of probability vectors are constructed and justified. Lumping all states of the Markov chain in each subspace into a single state, an aggregated process is shown to converge to a continuous-time Markov chain whose generator is an average...

Additive Functionals for Discrete-time Markov Chains with Applications to Birth–death Processes

In this paper we are interested in bounding or calculating the additive functionals of the first return time on a set for discrete-time Markov chains on a countable state space, which is motivated by investigating ergodic theory and central limit theorems. To do so, we introduce the theory of the minimal nonnegative solution. This theory combined with some other techniques is proved useful for ...