Current point-based planning algorithms for solving partially observable Markov decision processes (POMDPs) have demonstrated that a good approximation of the value function can be derived by interpolation from the values of a specially selected set of points. The performance of these algorithms can be improved by eliminating unnecessary backups or concentrating on more important points in the ...

The ever increasing capabilities and complexity of sensor networks have led to an increased interest in sensor placement and observation planning problems. Many sensor placement and planning problems, however, lead to instances of the intractable classical planning problems or (similarly intractable) partially observable Markov decision processes. We consider the problem of planning sensor acti...

Both humans and the sensors on an autonomous vehicle have limited sensing capabilities. When these limitations coincide with scenarios involving vulnerable road users, it becomes important to account for these limitations in the motion planner. For the scenario of an occluded pedestrian crosswalk, the speed of the approaching vehicle should be a function of the amount of uncertainty on the road...

We propose a node-removal/arc-reversal algorithm for influence diagram evaluation that includes reductions that allow an influence diagram to be solved by a generalization of the dynamic programming approach to solving partially observable Markov decision processes (POMDPs). Among its potential advantages, the algorithm allows a more flexible ordering of node removals, and a POMDPinspired appro...

We report the ‘Recurrent Deterioration’ (RD) phenomenon observed in online recommender systems. The RD phenomenon is reflected by the trend of performance degradation when the recommendation model is always trained based on users’ feedbacks of the previous recommendations. There are several reasons for the recommender systems to encounter the RD phenomenon, including the lack of negative traini...

DECISION-THEORETIC META-REASONING IN PARTIALLY OBSERVABLE AND DECENTRALIZED SETTINGS

Decision processes with incomplete state feedback have been traditionally modeled as Partially Observable Markov Decision Processes. In this paper, we present an alternative formulation based on probabilistic regular languages. The proposed approach generalizes the recently reported work on language measure theoretic optimal control for perfectly observable situations and shows that such a fram...

We present a new algorithm that extends the Reinforcement Learning framework to Partially Observed Markov Decision Processes (POMDP). The main idea of our method is to build a state extension, called exhaustive observable, which allow us to define a next processus that is Markovian. We bring the proof that solving this new process, to which classical RL methods can be applied, brings an optimal...