Currently, simulation is usually used to estimate network degree distribution P(k) and to examine if a network model predicts a scale-free network when an analytical formula does not exist. An alternative Markovian chain-based numerical method was proposed by Shi [Phys. Rev. E 71, 036140 (2005)] to compute time-dependent degree distribution P(k,t) . Although the numerical results demonstrate a ...

We consider the joint pricing and inventory control problem for a single product with a finite horizon and periodic review. The demand distribution in each period is determined by an exogenous Markov chain. Pricing and ordering decisions are made at the beginning of each period and all shortages are backlogged. The surplus costs as well as fixed and variable costs are state dependent. We show t...

We propose a mapping of spreading dynamics to an ensemble of weighted networks, where edge weights represent propagation time delays. In this mapping, shortest paths in the weighted networks preserve the temporal causality of spreading. Furthermore, for efficient sampling, we construct a Markov Chain (Gibbs sampler) over elements of an ensemble of mapped weighted networks. Our framework provide...

The wound process in patients with diabetes mellitus in addition to VAC-therapy (Vacuum-assisted closure — VAC® therapy) is examined. The model of the biological system describing the wound process is built on three independently functioning subsystems; it is assumed to be ergodic and presented with a symmetric graph. The mathematical model of the wound process dynamics is constructed based on ...

A problem of robust guaranteed cost control of stochastic discrete-time systems with parametric uncertainties under Markovian switching is considered. The jump Markovian switching is modelled by a discrete-time Markov chain and the noise or stochastic environmental disturbance is modelled by a sequence of identically independently normally distributed random variables. Using linear matrix inequ...

In this paper we propose a novel MATLAB tool, called Q-MAM, to compute queue length, waiting time and sojourn time distributions of various discrete and continuous time queuing systems with an underlying structured Markov chain/process. The underlying paradigms include M/G/1and GI/M/1-type, quasi-birth-death and non-skip-free Markov chains (implemented by the SMCSolver tool), as well as Markov ...

The configuration-controlled regime and the diffusion-controlled regime of conformation-modulated fluorescence emission are systematically studied for Markovian and non-Markovian dynamics of the reaction coordinate. A path integral simulation is used to model fluorescence quenching processes on a semiflexible chain. First-order inhomogeneous cumulant expansion in the configuration-controlled re...

In this paper we present an iterative technique based on Monte Carlo simulations for deriving the optimal control of the infinite horizon linear regulator problem of discretetime Markovian jump linear systems for the case in which the transition probability matrix of the Markov chain is not known. It is well known that the optimal control of this problem is given in terms of the maximal solutio...

We compute exactly the distribution of the occupation time in a discrete non-Markovian toy sequence that appears in various physical contexts such as the diffusion processes and Ising spin glass chains. The non-Markovian property makes the results nontrivial even for this toy sequence. The distribution is shown to have non-Gaussian tails characterized by a nontrivial large deviation function th...

A Markovian Agent Model (MAM) is a stochastic model that provides a flexible, powerful and scalable way for analyzing complex systems of distributed interacting objects. The constituting bricks of a MAM are the Markovian Agents (MA) represented by a finite state continuous time Markov chain (CTMC) whose infinitesimal generator is composed by a fixed component (the local behaviour) and an induce...