We consider a finite horizon Markov decision process with only terminal rewards. We describe a finite algorithm for computing a Markov deterministic policy which maximises the variance penalised reward and we outline a vertex elimination algorithm which can reduce the computation involved.

This paper models imperfect inspection policies for a system with a finite operational time requirement. Two types of maintenance policy are considered: that in which an alarm (a positive inspection) is followed by a check of the validity of the alarm at additional cost, and if the system is good it continues in service and if failed it is retired; and that in which a positive inspection leads ...

Graph planning gives rise to fundamental algorithmic questions such as shortest path, traveling salesman problem, etc. A classical problem in discrete planning is to consider a weighted graph and construct a path that maximizes the sum of weights for a given time horizon T . However, in many scenarios, the time horizon is not fixed, but the stopping time is chosen according to some distribution...

Graph and automata theory are widely used in the algorithmic solution of fundamental problems in verification. Verification problems often boil down to solving reachability questions in graphs, optimization problems, and combinatorial problems in automata. Fundamental algorithmic problems in computer science have been well studied, such as the shortest path problem, the travelling salesman prob...

The accuracy of an identified model depends on the choice of input signal. Persistency of excitation is a necessary criterion for such signals. In this paper we develop additional criteria for input signal selection, in particular, the input at each time step is chosen to minimize the predicted variance of the system estimate at the next time step. We extend the method to the finitehorizon inpu...

“Human computation” is a recent approach that extracts information from large numbers of Web users. reCAPTCHA is a human computation project that improves the process of digitizing books by getting humans to read words that are difficult for OCR algorithms to read (von Ahn et al., 2008). In this paper, we address an interesting strategic control problem inspired by the reCAPTCHA project: given ...

In this survey, receding horizon control is presented as a method for obtaining approximately optimal solutions to infinite horizon optimal control problems by iteratively solving a sequence of finite horizon optimal control problems. We investigate conditions under which we can obtain mathematically rigorous approximation results for this approach. A key ingredient of our analysis is the so-ca...

The use of Chebyshev polynomials in solving finite horizon optimal control problems associated with general linear time-varying systems with constant delay is well known in the literature. The technique is modified in the present paper for the finite horizon control of dynamical systems with time periodic coefficients and constant delay. The governing differential equations of motion are conver...

the inventory routing problem (irp) arises in the context of vendor-managed systems. this problem addresses jointly solving an inventory management problem and a vehicle routing problem. in this paper, we consider a multi-depot multi-vehicle inventory routing problem in which suppliers produce one type of product to be delivered to customers during a finite time horizon. we propose a two-phase ...

This paper deals with unconstrained receding horizon control of nonlinear systems with a general, nonnegative terminal cost. Earlier results have indicated that when the terminal cost is a suitable local control Lyapunov function, the receding horizon scheme is stabilizing for any horizon length. In a recent paper, the authors show that there always exist a uniform horizon length which guarante...