We consider a distributionally robust partially observable Markov decision process (DR-POMDP), where the distribution of transition-observation probabilities is unknown at beginning each period, but their realizations can be inferred using side information end period after an action being taken. build ambiguity set joint bounded moments via conic constraints and seek optimal policy to maximize ...

the notion of infinite order markov process is introduced and the markov property of the flow of information is established.

We study Probabilistic Workflow Nets (PWNs), a model extending van der Aalst’s workflow nets with probabilities. We give a semantics for PWNs in terms of Markov Decision Processes and introduce a reward model. Using a result by Varacca and Nielsen, we show that the expected reward of a complete execution of the PWN is independent of the scheduler. Extending previous work on reduction of non-pro...

For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the original process when restricting to the approximating set. The embedded process can be used as an approximation which, being finite, is more convenient for comput...

This paper presents an adaptive tutoring system based on a Markov Decision Process model. Applying this model to a tutoring system is a unique concept in computer based training. After a detailed derivation of the model, results from experiments are presented that show the Markov Decision Process model is an effective tool for adding intelligence to computer based tutoring.

During the last decades, collaborative robots capable of operating out their cages are widely used in industry to assist humans mundane and harsh manufacturing tasks. Although such inherently safe by design, they commonly accompanied external sensors other cyber-physical systems, facilitate close cooperation with humans, which frequently render ecosystem unsafe prone hazards. We introduce a met...