Asymptotic behaviors of aggregated Markov processes

A Canonical Representation for Aggregated Markov Processes

A deterministic function of a Markov process is called an aggregated Markov process. We give necessary and suucient conditions for the equivalence of continuous-time aggregated Markov processes. For both discrete-and continuous-time, we show that any aggregated Markov process which satisses mild regularity conditions can be directly converted to a canonical representation which is unique for ea...

Aggregated Markov Processes and Channel Gating Kinetics

A finite state Markov process is aggregated into several groups. Rather than observing the underlying Markov process. one is only able to observe the aggregated process. What can be learned about the underlying process from the aggregated one? Such questions arise in she study of gating mechanisms in ion channels in muscle and nerve cell membranes. We discuss some recent results and their impli...

Asymptotic properties of constrained Markov Decision Processes

We present in this paper several asymptotic properties of constrained Markov Decision Processes (MDPs) with a countable state space. We treat both the discounted and the expected average cost, with unbounded cost. We are interested in (1) the convergence of nite horizon MDPs to the innnite horizon MDP, (2) convergence of MDPs with a truncated state space to the problem with innnite state space,...

An Asymptotic Simplex Method and Markov Decision Processes

Asymptotic Behavior of Multivariate Reward Processes with Nonlinear Reward Functions