for an m-state homogeneous irreducible Markov chain with transition probability matrix Qm×m is known, but the chain requires updating by altering some of its transition probabilities or by adding or deleting some states. Suppose that the updated transition probability matrix Pn×n is also irreducible. The updating problem is to compute the updated stationary distribution π = (π1, π2, . . . , πn)...

The model A hidden Markov model is characterized by a set of M states, by an initial probability distribution for the first state, by a transition probability matrix linking successive states, and by a state-dependent probability distribution on the outputs. We represent the state at time t as a multinomial random variable qt, with components q t, for i = 1, . . . ,M . Thus q t is equal to one ...

* Research supported in part by National Nature Science Foundation of China Abstract A suboptimal approach to the d( 0 d ≥ ) step fixed-lag smoothing problem for Markovian switching systems is presented. Multiple Model Estimation techniques have been widely used in solving state estimation problems of these systems. We demonstrated that the mode probability of each fixed-lag smoother at time k-...

• The main idea is to obtain a generalized tensor description of the transition probability matrix or the transition rate matrix • N finite automata. One automaton is used to model one component. • The state space is included into the Cartesian product of the state space of the automata.

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t) and conservative matrix Q(t) = [qij(t)] of nonhomogeneous transition rates qij(t), and use it to construct the transition ...

As discussed in previous lecture, a process that is both Markov and time independent is called a stationary Markov Process. A finite state stationary Markov process {Xi} can be viewed as a Markov chain. Fig. 1 shows a two-state Markov chain diagram. In this example, there are two states labeled as a and b. Thus, the values of Xi ∈ [a, b]. Associated with a Markov chain is a transition probabili...

Probabilistic Boolean Network (PBN) is widely used to model genetic regulatory networks. Evolution of the PBN is according to the transition probability matrix. Steady-state (long-run behaviour) analysis is a key aspect in studying the dynamics of genetic regulatory networks. In this paper, an efficient method to construct the sparse transition probability matrix is proposed, and the power meth...