This paper studies the problem of diffusion in Markovian models, such as hidden Markov models (HMMs) and how it makes very difficult the task of learning of long-term dependencies in sequences. Using results from Markov chain theory, we show that the problem of diffusion is reduced if the transition probabilities approach 0 or 1. Under this condition, standard HMMs have very limited modeling ca...

A phase-type distribution is the distribution of hitting time in a finite-state Markov chain. A phase-type distribution is triangular if there exist an upper triangular markovian representation. We introduce in this paper an extension of the triangular phase-type distributions, which we call monocyclic distributions. They are convolutions of Erlang and feedback Erlang distributions. We will sho...

This paper studies the problem of di usion in Markovian models such as hidden Markov models HMMs and how it makes very di cult the task of learning of long termdependencies in sequences Using results from Markov chain theory we show that the problem of di usion is reduced if the transition probabilities approach or Under this condition standard HMMs have very limited modeling capabilities but i...

This paper is concerned with long-run average cost minimization of a stochastic inventory problem with Markovian demand, fixed ordering cost, convex surplus cost, and lost sales. The states of the Markov chain represent different possible states of the environment. Using a vanishing discount approach, a dynamic programming equation and the corresponding verification theorem are established. Fin...

Addressed is the problem of state estimation for dynamic Markovian jump systems (MJS) with unknown transitional probability matrix (TPM) of the embedded Markov chain governing the system jumps. Based on recent authors’ results, proposed is a new TPM-estimation algorithm that utilizes stochastic simulation methods (viz. Bayesian sampling) for finite mixtures’ estimation. Monte Carlo simulation r...

This paper studies the problem of diiusion in Markovian models, such as hidden Markov models (HMMs) and how it makes very diicult the task of learning of long-term dependencies in sequences. Using results from Markov chain theory, we show that the problem of diiusion is reduced if the transition probabilities approach 0 or 1. Under this condition, standard HMMs have very limited modeling capabi...

Owing to their theoretical and practical significance, (1) has received great attention and has been recently studied extensively, andwe heremention Skorokhod [1] andMao and Yuan [2] among many others. However, in the real world, the condition that coefficients f and g in (1) are perturbed by the same Markovian chain is too restrictive. For example, in the classical Black-Scholes model, the ass...

This paper is concerned with long-run average cost minimization of a stochastic inventory problem with Markovian demand, xed ordering cost, and convex surplus cost. The states of the Markov chain represent diierent possible states of the environment. Using a vanishing discount approach, a dynamic programming equation and the corresponding veriication theorem are established. Finally, the existe...

This paper studies the problem of diiusion in Markovian models, such as hidden Markov models (HMMs) and how it makes very diicult the task of learning of long-term dependencies in sequences. Using results from Markov chain theory, we show that the problem of diiusion is reduced if the transition probabilities approach 0 or 1. Under this condition, standard HMMs have very limited modeling capabi...

We give a survey on some recent developments in the spectral theory of transfer operators, also called Ruelle-Perron-Frobenius (RPF) operators, associated to expanding and mixing dynamical systems. Different methods for spectral study are presented. Topics include maximal eigenvalue of RPF operators, smooth invariant measures, ergodic theory for chain of markovian projections, equilibrium state...