Motivated by problems arising in time-dependent queues and dynamic systems with random environment, this work develops moderate deviations principles for dynamic systems driven by a fast-varying nonhomogeneous Markov chain in continuous time. A distinct feature is that the Markov chain is time dependent or inhomogeneous so are the dynamic systems. Under irreducibility of the nonhomogeneous Mark...

In this paper, we consider the problem of optimizing the quantiles of the cumulative rewards of Markov Decision Processes (MDP), to which we refers as Quantile Markov Decision Processes (QMDP). Traditionally, the goal of a Markov Decision Process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly to be infinite). In many applications, however, a decision maker may ...

‎in the present paper we investigate the $l_1$-weak ergodicity of‎ ‎nonhomogeneous continuous-time markov processes with general state‎ ‎spaces‎. ‎we provide a necessary and sufficient condition for such‎ ‎processes to satisfy the $l_1$-weak ergodicity‎. ‎moreover‎, ‎we apply‎ ‎the obtained results to establish $l_1$-weak ergodicity of quadratic‎ ‎stochastic processes‎.

When the process underlying DNA substitutions varies across evolutionary history, some standard Markov models underlying phylogenetic methods are mathematically inconsistent. The most prominent example is the general time-reversible model (GTR) together with some, but not all, of its submodels. To rectify this deficiency, nonhomogeneous Lie Markov models have been identified as the class of mod...