منابع مشابه
Time-Sharing Policies for Controlled Markov Chains
We propose a class of non-stationary policies called \policy time sharing" (p.t.s.), which possess several desirable properties for problems where the criteria are of the average-cost type; an optimal policy exists within this class, the computation of optimal policies is straightforward, and the implementation of this policy is easy. While in the nite state case stationary policies are also kn...
متن کاملControlled Markov Chains, Graphs, and Hamiltonicity
This manuscript summarizes a line of research that maps certain classical problems of discrete mathematics — such as the Hamiltonian Cycle and the Traveling Salesman Problems — into convex domains where continuum analysis can be carried out. Arguably, the inherent difficulty of these, now classical, problems stems precisely from the discrete nature of domains in which these problems are posed. ...
متن کاملControlled Markov chains with safety upper bound
In this paper we introduce and study the notion of safety control of stochastic discrete event systems (DESs), modeled as controlled Markov chains. For non-stochastic DESs, modeled by state machines or automata, safety is specified as a set of forbidden states, or equivalently by a binary valued vector that imposes an upper bound on the set of states permitted to be visited. We generalize this ...
متن کاملEmpirical Bayes Estimation in Nonstationary Markov chains
Estimation procedures for nonstationary Markov chains appear to be relatively sparse. This work introduces empirical Bayes estimators for the transition probability matrix of a finite nonstationary Markov chain. The data are assumed to be of a panel study type in which each data set consists of a sequence of observations on N>=2 independent and identically dis...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: The Annals of Probability
سال: 1975
ISSN: 0091-1798
DOI: 10.1214/aop/1176996445