In this paper, absorbing Markov chain models are developed to determine the optimum process mean levels for both a single-stage and a serial two-stage production system in which items are inspected for conformity with their specification limits. When the value of the quality characteristic of an item falls below a lower limit, the item is scrapped. If it falls above an upper limit, the item is ...

We examine stochastic coarse-graining strategies for two biomolecular systems. First, we compute the large-scale transport properties of the basic flashing ratchet mathematical model for (Brownian) molecular motors and consider in this light whether the underlying continuous-space, continuous-time Markovian model can be coarse-grained as a discrete-state, continuous-time Markovian random walk m...

A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decisiontheoretic planning, where many desirable behaviours are more naturally expressed as properties of execution sequences rather than as properties of states, NMRDPs form a more natural model than the commonly adopted fully Markovi...

The Hamiltonian approach of the Pauli-Fierz model is studied by Derezinski and Jaksic (cf. [7]). In this paper, we give the Markovian description of this model. Using the weak coupling limit, we derive the quantum Markovian semigroup of the Pauli-Fierz system from its Hamiltonian description. Also, we give the explicit form of the associated Lindblad generator. As a consequence, we study the pr...

The behavioral equivalence that is typically used to relate Markovian process terms and to reduce their underlying state spaces is Markovian bisimilarity. One of the reasons is that Markovian bisimilarity is consistent with ordinary lumping. The latter is an aggregation for Markov chains that is exact, hence it guarantees the preservation of the performance characteristics across Markovian bisi...

We introduce a new model for rill erosion. We start with a network similar to that in the Discrete Web** and instantiate a dynamics which makes the process highly non-Markovian. The behavior of nodes in the streams is similar to the behavior of Polya urns with time-dependent input. In this paper we use a combination of rigorous arguments and simulation results.

The problem of optimizing Markovian models with infinitely or finite but infeasible large state space is considered. In several practically interesting cases the state space of the model is finite and extremely large or infinite, and the transition and decision structures have some regular property which can be exploited for efficient analysis and optimization. Among the Markovian models with r...

This article aims at illustrating the generality and the powerfulness of Bayesian and specially Markovian frameworks for different remote sensing applications and in particular for SAR (Synthetic Aperture Radar) image processing. Indeed, the Markovian model is a very convenient way to introduce prior knowledge on the problem to solve. It will first be evoked with examples on the pixel level lik...

This article aims at illustrating the powerfulness of Bayesian and specially Markovian frameworks for different remote sensing applications and in particular for SAR (Synthetic Aperture Radar) image processing. Indeed, the Markovian model is a very convenient way to introduce prior knowledge on the problem to solve. It will first be evoked with examples on the pixel level like filtering, segmen...

Lahar: Warehousing Markovian Streams Julia Maureen Letchner Chair of the Supervisory Committee: Professor Magdalena Balazinska Computer Science and Engineering A huge amount of the world’s data is both sequential and low-level. Many applications consume higher-level information, such as words and sentences, that is inferred from low-level sequences such as raw audio signals using a model (e.g.,...