This note presents a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. produces an auxiliary PDMDP from original one. The possesses certain desired properties, which may not be possessed by PDMDP. We apply this to risk-sensitive PDMDPs total cost criteria, comment on its connection uni...

A technique is developed for comparing a non-Markov process to a Markov process on a general state space with many possible stochastic orderings. Two such comparisons with a common Markov process yield a comparison between two non-Markov processes. The technique, which is based on stochastic monotonidty of the Markov process, yields stochastic comparisons of the limiting distributions and the m...

The Baum-Welch algorithm is a technique for the maximum likelihood parameter estimation of probabilistic functions of Markov processes. We apply this technique to nonstationary Markov processes and explore a relationship between the Baum-Welch algorithm and the BCJR algorithm. Furthermore, we apply the Baum-Welch algorithm to two nonstationary Markov processes and obtain the turbo decoding algo...

We give a new algorithm for the genotype phasing problem. Our solution is based on a hidden Markov model for haplotypes. The model has a uniform structure, unlike most solutions proposed so far that model recombinations using haplotype blocks. In our model, the haplotypes can be seen as a result of iterated recombinations applied on a few founder haplotypes. We find maximum likelihood model of ...

This paper is intended to provide a numerical algorithm based on random sampling for solving the linear Volterra integral equations of the second kind. This method is a Monte Carlo (MC) method based on the simulation of a continuous Markov chain. To illustrate the usefulness of this technique we apply it to a test problem. Numerical results are performed in order to show the efficiency and accu...