Providing probabilistic analysis of queueing models can be difficult when the input distributions are non-Markovian. In response, a plethora of methods have been developed to approximate a general renewal process by a process with the time between renewals being distributed as a phase type random variable, which allows the resulting queueing models to become analytically or numerically tractabl...

A rigorous formalism for the extraction of state-to-state transition functions from a Boltzmann-weighted ensemble of microcanonical molecular dynamics simulations has been developed as a way to study the kinetics of protein folding in the context of a Markov chain. Analysis of these transition functions for signatures of Markovian behavior is described. The method has been applied to an example...

This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems — like large-scale computers, client-server architectures, networks — can accurately be described using such stochastic specification formalisms. The main...

A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decisiontheoretic planning, where many desirable behaviours are more naturally expressed as properties of execution sequences rather than as properties of states, NMRDPs form a more natural model than the commonly adopted fully Markovi...

This article describes the batch Markovian arrival process (BMAP), a point process that is characterized by Markov-modulated batch arrivals of random size. The BMAP is a generalization of many well-known processes including the Markovian arrival process (MAP), the Poisson process, and the Markov-modulated Poisson process. It provides a common framework for modeling arrival processes in a variet...

We introduce Markovian cocycle perturbations of the groups of transformations associated with the classical and quantum stochastic processes with stationary increments, which are characterized by a localization of the perturbation to the algebra of events of the past. It is namely the definition one needs because the Markovian perturbations of the Kolmogorov flows associated with the classical ...