منابع مشابه
Path Decompositions for Markov Chains
We present two path decompositions of Markov chains (with general state space) by means of harmonic functions, which are dual to each other. They can be seen as a generalization of Williams' decomposition of a Brownian motion with drift. The results may be illustrated by a multitude of examples, but we confine ourselves to different types of random walks and the Pólya urn. 1. Introduction and m...
متن کاملPath Integrals for Continuous-Time Markov Chains
This note presents a method of evaluating the distribution of a path integral for Markov chains on a countable state space. We illustrate the method with reference to several models, including birth-death processes and the birth, death and catastrophe process.
متن کاملResistors, Markov Chains & Dynamic Path Planning
Dynamic planning involves continuously updating a map by sensing changes in the environment and planning appropriate actions, with all tasks sharing common computational resources. We use harmonic functions for dynamic planning. Analogous representations of harmonic functions as Markov chains and resistor networks are used to develop the notion of escape probability and energy dissipation. Thes...
متن کاملTaylor Expansion for the Entropy Rate of Hidden Markov Chains
We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...
متن کاملThe Rate of Rényi Entropy for Irreducible Markov Chains
In this paper, we obtain the Rényi entropy rate for irreducible-aperiodic Markov chains with countable state space, using the theory of countable nonnegative matrices. We also obtain the bound for the rate of Rényi entropy of an irreducible Markov chain. Finally, we show that the bound for the Rényi entropy rate is the Shannon entropy rate.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: The Annals of Probability
سال: 2004
ISSN: 0091-1798
DOI: 10.1214/009117904000000234