This paper derives a model for the profitability of credit cards, which allow lenders to find the optimal dynamic credit limit policy. The model is a Markov decision process, where the states of the system are based on the borrower’s behavioural score and the decisions are what credit limit to give the borrower each period. In determining the Markov chain which best describes the borrower’s per...

Partially Observable Markov Decision Processes (POMDPs) have been applied extensively to planning in environments where knowledge of an underlying process is confounded by unknown factors[3, 4, 7]. By applying the POMDP architecture to a basic recognition task, we introduce a novel pattern recognizer that operates under partially observable conditions. This Single Action Partially Observable Ma...

Partially observable Markov decision processes (POMDP) are well-suited for realizing sequential decision making capabilities that respect uncertainty in Companion systems that are to naturally interact with and assist human users. Unfortunately, their complexity prohibits modeling the entire Companion system as a POMDP. We therefore propose an approach that makes use of abstraction to enable em...

This paper examines the problem of finding an optimal policy for a Partially Observable Markov Decision Process (POMDP) when the model is not known or is only poorly specified. We propose two formulations of the problem. The first formulation relies on a model of the uncertainty that is added directly into the POMDP planning problem. This has some interesting theoretical properties, but is impr...

We consider a Markov decision process model of a network revenue management problem. Working within this formal framework, we study policies that combine aspects of mathematical programming approaches and pure Markov decision process methods. The policies employ heuristics early in the booking horizon, and switch to a more-detailed decision rule closer to the time of departure. We present a fam...

In this paper we discuss the modelling of diagnosis and treatment of diseases through the use of Partially Observable Markov Decision Processes, with the possibility of optimizing any criteria that may be quantified. Such model could be used in the making of a tool for aiding physicians with a second opinion.

The only available therapy for patients with end-stage liver disease is organ transplantation. In the United States, patients with end-stage liver disease are placed on a waiting list and offered livers based on location and waiting time, as well as current and past health. Although there is a shortage of cadaveric livers, 45% of all cadaveric liver offers are declined by the first transplant s...

The contribution of the UvA Rescue Team is an attempt to lay a theoretical foundation by describing the planning and coordination problem formally as an POMDP problem, which will allow to apply POMDP-solution methods in this application area. To be able to solve the POMDP problem for large state spaces and long planning histories, our team has chosen for an approximation of the Dec-POMDPs throu...