The probable lack of some arcs and nodes in the stochastic networks is considered in this paper, and its effect is shown as the arrival probability from a given source node to a given sink node. A discrete time Markov chain with an absorbing state is established in a directed acyclic network. Then, the probability of transition from the initial state to the absorbing state is computed. It is as...

Investors use different approaches to select optimal portfolio. so, Optimal investment choices according to return can be interpreted in different models. The traditional approach to allocate portfolio selection called a mean - variance explains. Another approach is Markov chain. Markov chain is a random process without memory. This means that the conditional probability distribution of the nex...

The arc costs are assumed to be online parameters of the network and decisions should be made while the costs of arcs are not known. The policies determine the permitted nodes and arcs to traverse and they are generally defined according to the departure nodes of the current policy nodes. In on-line created tours arc costs are not available for decision makers. The on-line traversed nodes are f...

In the last lecture we got familiar with the concept of discrete-time Markov chains and Hidden Markov Models (HMMs). A Markov chain is a discrete random process that abides by the Markov property, that the probability of the next state depends only on the current state and not the past. The Markov chain models how a state changes from step to step using transition probabilities. Therefore, a Ma...

We consider an extension of Discrete Time Markov Chain queueing model to the quantum domain by use of Discrete Time Quantum Markov Chain. We introduce methods for numerical analysis of such models. Using this tools we show that quantum model behaves fundamentally differently from the classical one.

We consider weak lumpability of denumerable discrete or continuous time Markov chains. Firstly, we are concerned with irreducible recurrent positive and R-positive Markov chains evolving in discrete time. We study the properties of the set of all initial distributions of the starting chain leading to an aggregated homogeneous Markov chain with respect to a partition of the state space. In parti...

the problem we investigate deals with the optimal assignment of resources to the activities of a stochastic project network. we seek to minimize the expected cost of the project include sum of resource utilization costs and lateness costs. we assume that the work content required by the activities follows an exponential distribution. the decision variables of the model are the allocated resourc...

'Pae multiple time scale decomposition of discrete time, finite state Markov chains is addressed. In [1, 2], the behavior of a continuous time Markov chain is approximated using a fast time scale, e-independent, continuous time process, and a reduced order perturbed process. The procedure can then be iterated to obtain a complete multiple time scale decomposition. In the discrete time case pres...