Hamiltonian Cycles and Singularly Perturbed Markov Chains
نویسندگان
چکیده
We consider the Hamiltonian cycle problem embedded in a singularly perturbed Markov decision process. We also consider a functional on the space of deterministic policies of the process that consists of the (1,1)-entry of the fundamental matrices of the Markov chains induced by the same policies. We show that when the perturbation parameter, ε, is less than or equal to 1 N2 the Hamiltonian cycles of the directed graph are precisely the minimizers of our functional over the space of deterministic policies. In the process, we derive analytical expressions for the possible N distinct values of the functional over the, typically, much larger space of deterministic policies.
منابع مشابه
Proof of the Hamiltonicity-trace Conjecture for Singularly Perturbed Markov Chains
We prove the conjecture formulated in Litvak and Ejov (2009), namely, that the trace of the fundamental matrix of a singularly perturbed Markov chain that corresponds to a stochastic policy feasible for a given graph is minimised at policies corresponding to Hamiltonian cycles.
متن کاملGeometric Interpretation of Hamiltonian Cycles Problem via Singularly Perturbed Markov Decision Processes
We consider the Hamiltonian cycle problem (HCP) embedded in a singularly perturbed Markov decision process (MDP). More specifically, we consider the HCP as an optimization problem over the space of long-run state-action frequencies induced by the MDP’s stationary policies. We also consider two quadratic functionals over the same space. We show that when the perturbation parameter, ", is suffici...
متن کاملSingularly Perturbed Finite Markov Chains with General Ergodic Structure
We analyse singularly perturbed Markov chains. Most previous research has been done under the assumption that the perturbed Markov chain is either ergodic or unichain. In this paper we do not impose any restrictions on the ergodic structure of the perturbed chain. The present approach is based on the inversion of analytic matrix-valued functions.
متن کاملDirected graphs, Hamiltonicity and doubly stochastic matrices
We consider the Hamiltonian cycle problem embedded in singularly perturbed (controlled)Markov chains. We also consider a functional on the space of stationary policies of the process that consists of the (1,1)-entry of the fundamental matrices of the Markov chains induced by the same policies. In particular, we focus on the subset of these policies that induce doubly stochastic probability tran...
متن کاملErgodic Control of a Singularly Perturbed Markov Process in Discrete Time with General State and Compact Action Spaces
Ergodic control of singularly perturbed Markov chains with general state and compact action spaces is considered. A new method is given for characterization of the limit of invariant measures, for perturbed chains, when the perturbation parameter goes to zero. It is also demonstrated that the limit control principle is satisfied under natural ergodicity assumptions about controlled Markov chain...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Math. Oper. Res.
دوره 29 شماره
صفحات -
تاریخ انتشار 2004