Control of Diffusions via Linear Programming

In this chapter we present an approach that leverages linear programming to approximate optimal policies for controlled diffusion processes, possibly with high-dimensional state and action spaces. The approach fits a linear combination of basis functions to the dynamic programming value function; the resulting approximation guides control decisions. Linear programming is used here to compute basis function weights. What we present extends the linear programming approach to approximate dynamic programming, previously developed in the context of discretetime stochastic control [19, 20, 7, 8, 9]. One might question the practical merits of such an extension relative to descretizing continuous-time models and treating them using previously developed methods. As will be made clear in this chapter, there are indeed important advantages in the simplicity and efficiency of computational methods made possible by working directly with a diffusion model. We begin in Section 1.1 by presenting a problem formulation and a linear programming characterization of optimal solutions. The numbers of variables