A ug 2 00 2 Local Solutions of the Dynamic Programming Equations and the Hamilton Jacobi Bellman PDE

We present a method for the solution of the Dynamic Programming Equation (DPE) that arises in an infinite horizon optimal control problem. The method extends to the discrete time version of Al’brecht’s procedure for locally approximating the solution of the Hamilton Jacobi Bellman PDE. Assuming that the dynamics and cost are C(R) and C(R) smooth, respectively, we explicitly find expansions of the optimal control and cost in a term by term fashion. These finite expansions are the first (r − 1)th and rth terms of the power series expansions of the optimal control and the optimal cost, respectively. Once the formal solutions to the DPE are found, we then prove the existence of smooth solutions to the DPE that has the same Taylor series expansions as the formal solutions. The Pontryagin Maximum Principle provides the nonlinear Hamiltonian dynamics with mixed direction of propagation and the conditions for a control to be optimal. We calculate the forward Hamiltonian dynamics, the dynamics that propagates forward in time. We learn the eigenstructure of the Hamiltonian matrix and symplectic properties which aid in finding the graph of the gradient of the optimal cost. Furthermore, the Local Stable Manifold Theorem, the Stokes’ Theorem, and the Implicit Function Theorem are some of the main tools used to show the optimal cost and the optimal control do exist and satisfy the DPE. Assuming that there is a forward Hamiltonian dynamics is to assume that the Hamiltonian matrix is invertible. If 0 is eigenvalue of the Hamiltonian matrix, then 0 is also a closed loop eigenvalue. We consider 0 as a possible closed loop eigenvalue since the optimal cost calculated term by term is true for all closed loop eigenvalues with magnitude less than 1. We prove that there exists a local stable manifold for