We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

this paper is devoted to the study of determining optimal process mean in system production with the two markets for the sale of goods. in this paper, we developed an absorbing markov chain model in production systems where all items are inspected %100 for conformance with their specification limits. when the value of the quality characteristic of an item falls below a lower limit, the item is ...

We present an exact dynamic programming solution for a finite-horizon decentralized two-player Markov decision process, where player 1 only has access to its own states, while player 2 has access to both player’s states but cannot affect player 1’s states. The solution is obtained by solving several centralized partially-observable Markov decision processes. We then conclude with several comput...

ive Meeting Summarization as a Markov Decision Process

The value 1 problem is a natural decision problem in algorithmic game theory. For partially observable Markov decision processes with reachability objective, this problem is defined as follows: are there observational strategies that achieve the reachability objective with probability arbitrarily close to 1? This problem was shown undecidable recently. Our contribution is to introduce a class o...

Computers have already eclipsed the level of human play in competitive Scrabble, but there remains room for improvement. In particular, there is much to be gained by incorporating information about the opponent’s tiles into the decision-making process. In this work, we quantify the value of knowing what letters the opponent has. We use observations from previous plays to predict what tiles our ...

In ci componioiz paper [l] we have developed a frame,work fo,r rating 01‘ comparing navigation packages. For a giuen enuir.onment a navigation package consists of (I motion planner and a sensor to be used during nauigation. The cibility to rate or measure a navigation package is aniportant in order to address issues like sensor custom%zution for an environment and choice of a m.otion plarumr in...

Good pedagogical actions are key components in all learning-teaching schemes. Automate that is an important Intelligent Tutoring Systems objective. We propose apply Partially Observable Markov Decision Process (POMDP) in order to obtain automatic and optimal pedagogical recommended action patterns in benefit of human students, in the context of Intelligent Tutoring System. To achieve that goal,...

We consider a special case of partially observable Markov decision processes that arises when state information is perfect but arrives with a delay. We rst formulate the decision process in its standard form and derive the Bellman equation that corresponds to it. We then introduce a second decision process that has a much simpler Bellman equation than the rst, and is therefore, in general, much...