We present a technique for speeding up the convergence of value iteration for par­ tially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incor­ porated into any existing POMDP value it­ eration algorithms. Experiments have been conducted on ...

We propose a simple technique for verifying probabilistic models whose transition probabilities are parametric. The key is to replace parametric transitions by nondeterministic choices of extremal values. Analysing the resulting parameter-free model using off-the-shelf means yields (refinable) lower and upper bounds on probabilities of regions in the parameter space. The technique outperforms t...

This paper presents an approach to score following. The realtime alignment of a performance with a score is obtained through the use of a hidden Markov model. The model works on two levels. The lower level compares the features of the incoming signal with the expected ones. Groups of states of the lower level are embedded in states at the higher level, which are used to model the performance by...

In this paper, we focus on queues with a superposition of periodic and bursty sources where the initial states of the sources are randomly and independently selected. The dynamics of the queue are described as a reducible Markov chain. For such queues, we consider the tail probability of the queue length as the performance measure and try to estimate it by simulations. To do simulation efficien...

Open queueing networks with bursty arrival processes, phase type service times, and losses are considered. A technique is proposed for the speciication of the models. Using such a speci-cation, a Markov chain can be generated automatically. Furthermore, this description can be used immediately for simulation. An approximate disaggregation{aggregation technique for nite state Markov chains is ap...

We present a novel technique to solve multiserver retrial systems with impatience. Unfortunately these systems do not present an exact analytic solution, so it is mandatory to resort to approximate techniques. This novel technique does not rely on the numerical solution of the steady-state Kolmogorov equations of the Continuous Time Markov Chain as it is common for this kind of systems but it c...

We propose a technique for decomposing and compressing the dataset in the parameter learning problem in Markov random fields. Our technique applies to incomplete datasets and exploits variables that are always observed in the given dataset. We show that our technique allows exact computation of the gradient and the likelihood, and can lead to orders-of-magnitude savings in learning time.

aspirin is one of the certified medicines commonly used for the secondary prevention of myocardial infarction (mi). aspirin side effects and gastrointestinal bleeding, in particular, have arisen debates on its use for the primary prevention of mi. the present research evaluates the cost-effectiveness of the use of aspirin in the primary prevention of mi among iranian men with average cardiovasc...