A Partially Observable Markovian Maintenance Process with Continuous Cost Functions

In this paper a two-state Markovian maintenance process where the true state is unknown will be considered. The operating cost per period is a continuous random variable which depends on the state of the process. If investigation cost is incurred at the beginning of any period, the system wit I be returned to the "in-control" state instantaneously. This problem is solved using the average criteria. The method involves exploiting the structure of the problem to develop an algorithm which is shown to be more efficient than the usual dynamic programming approach. Results of extensive tests show the accuracy of this algorithm. In addition, it is shown that if certain condition is satisfied, then it is possible to find the average cost per period by a simple calculation.