A discrete time stochastic feedback control system consisting of a non-linear plant, a sensor, a controller, and a noisy communication channel between the sensor and the controller is considered. The sensor has limited memory and at each time, it transmits an encoded symbol over the channel and updates its memory. The controller receives a noise-corrupted copy of the transmitted symbol. It gene...

A method is presented for direct trajectory optimization and costate estimation of finite-horizon and infinite-horizon optimal control problems using global collocation at Legendre-Gauss-Radau (LGR) points. A key feature of the method is that it provides an accurate way to map the KKT multipliers of the nonlinear programming problem to the costates of the optimal control problem. More precisely...

Models for long-term planning often lead to infinite horizon stochastic programs that offer significant challenges for computation. Finite-horizon approximations are often used in these cases but they may also become computationally difficult. In this paper, we directly solve for value functions of infinite horizon stochastic programs. We show that a successive linear approximation method conve...

Many control problems in environments that can be modeled as Markov decision processes (MDPs) concern infinite-time horizon specifications. The classical aim in this context is to compute a control policy that maximizes the probability of satisfying the specification. In many scenarios, there is however a non-zero probability of failure in every step of the system’s execution. For infinite-time...

The aim of this paper is to provide improvments to Pontryagin principles in infinite-horizon discrete-time framework when the space of states and of space of controls are infinite-dimensional. We use the method of reduction to finite horizon and several functional-analytic lemmas to realize our aim.

The norm is one of the fundamental concepts of linear algebra and functional analysis. The notion of the norm is often employed in engineering, e.g. in control engineering, where main application is calculating the norm of the transfer function. Unfortunately existing methods are applicable for systems that can be described using Laplace transform, i.e. linear time-invariant (LTI) systems. An o...

We prove a general theorem that the L2ρ(R ;R) ⊗ L2ρ(R ;R) valued solution of an infinite horizon backward doubly stochastic differential equation, if exists, gives the stationary solution of the corresponding stochastic partial differential equation. We prove the existence and uniqueness of the L2ρ(R ;R)⊗Lρ(R ;R) valued solutions for backward doubly stochastic differential equations on finite a...

We give a sufficient condition on the coefficients of a class of infinite horizon BDSDEs, under which the infinite horizon BDSDEs have a unique solution for any given square integrable terminal values. We also show continuous dependence theorem and convergence theorem for this kind of equations. A probabilistic interpretation for solutions to a class of stochastic partial differential equations...