Abstract Continuous time Markov chains are often used in the literature to model the dynamics of a system with low species count and uncertainty in transitions. In this paper, we investigate three particular algorithms that can be used to numerically simulate continuous time Markov chain models (a stochastic simulation algorithm, explicit and implicit tau-leaping algorithms). To compare these m...

A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We b...

Specialized classifiers, namely those dedicated to a subset of classes, are often adopted in realworld recognition systems. However, integrating such classifiers is nontrivial. Existing methods, e.g. weighted average, usually implicitly assume that all constituents of an ensemble cover the same set of classes. Such methods can produce misleading predictions when used to combine specialized clas...

The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix P (.) and ergodic matrix Π is the matrix D ≡ ∫∞ 0 (P (t)−Π)dt. We give conditions for D to exist and discuss properties and a representation of D. The deviation matrix of a birth-death process is investigated in detail. We also describe a new application of deviation matrices by showing that a me...

We develop a reversible jump Markov chain Monte Carlo approach to estimating the posterior distribution of phylogenies based on aligned DNA/RNA sequences under several hierarchical evolutionary models. Using a proper, yet nontruncated and uninformative prior, we demonstrate the advantages of the Bayesian approach to hypothesis testing and estimation in phylogenetics by comparing different model...

A variety of phenomena are best described using dynamical models which operate on a discrete state space and in continuous time. Examples include Markov (and semiMarkov) jump processes, continuous-time Bayesian networks, renewal processes and other point processes. These continuous-time, discrete-state models are ideal building blocks for Bayesian models in fields such as systems biology, genet...

This paper mainly analyzes the applications of the Generator matrices in a Continuous Time Markov Chain (CTMC). Hidden Markov models [HMMs] together with related probabilistic models such as Stochastic Context-Free Grammars [SCFGs] are the basis of many algorithms for the analysis of biological sequences. Combined with the continuous-time Markov chain theory of likelihood based phylogeny, stoch...

a r t i c l e i n f o a b s t r a c t MSC: 34D20 34K50 60H10 93E15 Keywords: Pure delay system Two-timescale Markov chain Stationary distribution Almost sure uniform stability Yorke's condition This work examines almost sure stability of a pure random delay system whose delay time is modeled by a finite state continuous-time Markov chain with two-time scales. The Markov chain contains a fast-va...

The aim of this thesis is to investigate the mathematics of Markov-modulated models for derivatives pricing. We consider a model where instantaneous stock volatility and drift are driven by a continuous time finite Markov chain. We present a new derivation of an integral representation for attainable non-path dependent options’ prices in a twostate and three-state Markov chain model, and comput...

The covariance ordering, for discrete and continuous time Markov chains, is defined and studied. This partial ordering gives a necessary and sufficient condition for MCMC estimators to have small asymptotic variance. Connections between this ordering, eigenvalues, and suprema of the spectrum of the Markov transition kernel, are provided. A representation of the asymptotic variance of MCMC estim...