This paper presents a method for optimal control of hybrid systems. An inequality of Bellman type is considered and every solution to this inequality gives a lower bound on the optimal value function. A discretization of this “hybrid Bellman inequality” leads to a convex optimization problem in terms of finitedimensional linear programming. From the solution of the discretized problem, a value ...

We exploit the separation of the …ltering and control aspects of quantum feedback control to consider the optimal control as a classical stochastic problem on the space of quantum states. We derive the corresponding Hamilton-Jacobi-Bellman equations using the elementary arguments of classical control theory and show that this is equivalent, in the Stratonovich calculus, to a stochastic Hamilton...

Neural Rewards Regression (NRR) is a generalisation of Temporal Difference Learning (TD-Learning) and Approximate Q-Iteration with Neural Networks. The method allows to trade between these two techniques as well as between approaching the fixed point of the Bellman iteration and minimising the Bellman residual. NRR explicitly finds a near-optimal Q-function without an algorithmic framework exce...

This paper shows that a finite-horizon version of the robust control criterion appearing in recent papers by Hansen, Sargent, and their coauthors can be described as recursive utility, which in continuous time takes the form of the Stochastic Differential Utility (SDU) of Duffie and Epstein (1992). While it has previously been noted that Bellman equations arising in robust control settings are ...

for x > 0. The maximizing (c = C(x), π= π(x)) satisfies V ′(x) = c−γ, π = − V ′(x) V ′′(x) (α− r1m)Σ = C(x) γC ′(x) (α− r1m)Σ, if V ′′ < 0, which will be checked later. Using an idea from Karatzas et al. [2], the Bellman equation (A.1) can be linearized by introducing a function X(c) := C−1(x). That is, the Bellman equation can be transformed into the the following equation κX ′′(c) = − − β − κ...

In this article, an efficient algorithm for an optimal decision strategy approximation is introduced. It approximates the Bellman equation without omitting the principal uncertainty stemming from incomplete knowledge. Thus, the approximated optimal strategy retains the ability to constantly verify the current knowledge. An integral part of the proposed solution is a reduction in memory demands ...

Convergence of corrections is examined for a predictorcorrector method to solve Bellman equations of multi-state stochastic optimal control in continuous time. Quadratic costs and constrained control are assumed. A heuristically linearized comparison equation makes the nonlinear, discontinuous Bellman equation amenable to linear convergence analysis. Convergence is studied using the Fourier sta...

Existing routing algorithms treat Quality of Service (QoS) parameters and secure routing as completely separate entities requiring separate algorithms. In this paper we propose secure QoS Distance Vector and secure Bellman-Ford-Moore routing algorithms that meet QoS requirements and satisfy security concerns. Security is achieved by placing filters in the network. The routing algorithms generat...