Local stability of Kolmogorov forward equations for finite state nonlinear Markov processes

A nonlinear fixed-lag smoother for finite-state Markov processes

The fixed-lag smoothing of random telegraph type signals is studied. The smoothers are derived by first obtaining fixed-point smoothing equations and then using a time discretization. Simulation results are described that verify the qualitative carry-over of known results for the l inear-Gaussian problem: the greater the lag, the greater the improvement; beyond a certain lag, no further improve...

Sufficiency of Markov Policies for Continuous-Time Markov Decision Processes and Solutions of Forward Kolmogorov Equation for Jump Markov Processes

In continuous-time Markov decision processes (CTMDPs) with Borel state and action spaces, unbounded transition rates, for an arbitrary policy, we construct a relaxed Markov policy such that the marginal distribution on the stateaction pairs at any time instant is the same for both the policies. This result implies the existence of a relaxed Markov policy that performs equally to an arbitrary po...

Nonlinear Markov processes and kinetic equations

Stability analysis of impulsive fuzzy differential equations with finite delayed state

In this paper we introduce some stability criteria for impulsive fuzzy system of differential equations with finite delay in states. Firstly, a new comparison principle for fuzzy differential system compared to crisp ordinary differential equation, based on a notion of upper quasi-monotone nondecreasing, in N dimentional state space is presented. Furthermore, in order to analyze the stability o...

Joint Hitting-Time Densities for Finite State Markov Processes

For a finite state Markov process and a finite collection {Γk, k ∈ K} of subsets of its state space, let τk be the first time the process visits the set Γk. We derive explicit/recursive formulas for the joint density and tail probabilities of the stopping times {τk, k ∈ K}. The formulas are natural generalizations of those associated with the jump times of a simple Poisson process. We give a nu...