In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of...

In the last lecture we got familiar with the concept of discrete-time Markov chains and Hidden Markov Models (HMMs). A Markov chain is a discrete random process that abides by the Markov property, that the probability of the next state depends only on the current state and not the past. The Markov chain models how a state changes from step to step using transition probabilities. Therefore, a Ma...

Recently, the Markov chain choice model has been introduced by Blanchet et al. to overcome the computational intractability for learning and revenue management for several modern choice models, including the mixed multinomial logit models. However, the known methods for learning the Markov models require almost all items to be offered in the learning stage, which is impractical. To address this...

This paper proposes a short-term water demand forecasting method based on the use of the Markov chain. This method provides estimates of future demands by calculating probabilities that the future demand value will fall within pre-assigned intervals covering the expected total variability. More specifically, two models based on homogeneous and non-homogeneous Markov chains were developed and pr...

abstract agricultural scientists have developed considerable interest in modeling and generation of rainfall as new ways of analyzing rainfall data and assessing its impact on agriculture. a combination of markov chain and gamma distribution function is recognized as a simple approach and is demonstrated to be effective in generating daily rainfall data for many environments. thus the availabil...

  In this paper Semi-Markov models are used to forecast the triple dimensions of next earthquake occurrences. Each earthquake can be investigated in three dimensions including temporal, spatial and magnitude. Semi-Markov models can be used for earthquake forecasting in each arbitrary area and each area can be divided into several zones. In Semi-Markov models each zone can be considered as a sta...

During the last decade, recommender systems became a popular class of models for many commercial websites. One of the best state-of-the-art methods for recommender systems are Matrix and Tensor Factorization models. Besides, Markov Chain models are common for representing sequential data problems (e.g. categorical time series data). The item recommendation problem of recommender systems in fact...

Latent Gaussian models are flexible models that are applied in several statistical applications. When posterior marginals or full conditional distributions in hierarchical Bayesian inference from these models are not available in closed form, Markov chain Monte Carlo methods are implemented. The component dependence of the latent field usually causes increase in computational time and divergenc...