in this paper at rst, a history of mathematical models is given.next, some basic information about random variables, stochastic processesand markov chains is introduced. as follows, the entropy for a discrete timemarkov process is mentioned. after that, the entropy for sis stochastic modelsis computed, and it is proved that an epidemic will be disappeared after a longtime.

The contribution gives a micro-structural insight into the pedestrian decision process during an egress situation. A method how to extract the decisions of pedestrians from the trajectories recorded during the experiments is introduced. The underlying Markov decision process is estimated using the finite mixture approximation. Furthermore, the results of this estimation can be used as an input ...

Reinforcement learning in nonstationary environments is generally regarded as an important and yet difficult problem. This paper partially addresses the problem by formalizing a subclass of nonsta-tionary environments. The environment model, called hidden-mode Markov decision process (HM-MDP), assumes that environmental changes are always confined to a small number of hidden modes. A mode basic...

0377-2217/$ see front matter 2011 Elsevier B.V. A doi:10.1016/j.ejor.2011.01.023 ⇑ Corresponding author. Tel.: +44 0 7796035843. E-mail addresses: [email protected] (M.M.C. So) Thomas). This paper derives a Markov decision process model for the profitability of credit cards, which allows lenders to find an optimal dynamic credit limit policy. The states of the system are based on the borrower’s ...

We associate a statistical vector to a trace and a geometrical embedding to a Markov Decision Process, based on a distance on words, and study basic Membership and Equivalence problems. The Membership problem for a trace w and a Markov Decision Process S decides if there exists a strategy on S which generates with high probability traces close to w. We prove that Membership of a trace is testab...