RICHARD BELLMAN

This paper presents a class of new approximate policy iteration algorithms for solving infinite-horizon, discounted Markov decision processes (MDPs) for which a model of the system is available. The algorithms are similar in spirit to Bellman residual minimization methods. However, by exploiting kernel-based regression techniques with nondegenerate kernel functions as the underlying cost-to-go ...

We study utility maximization for power utility random elds with and without intermediate consumption in a general semimartingale model with closed portfolio constraints. We show that any optimal strategy leads to a solution of the corresponding Bellman equation. The optimal strategies are described pointwise in terms of the opportunity process, which is characterized as the minimal solution of...

Evolutionary programming is a stochastic optimization procedure which has proved useful in optimizing difficult functions. It is shown that evolutionary programing can be used to solve the Bellman equation problem with a high degree of accuracy and substantially less CPU time than Bellman equation iteration. Future applications will focus on sometimes binding constraints – a class of problem fo...

We obtain Fourier inequalities in the weighted Lp spaces for any 1<p<∞ involving Hardy–Cesàro and Hardy–Bellman operators. extend these results to product Hardy p⩽1. Moreover, boundedness of Hardy-Cesàro Hardy-Bellman operators various (Lebesgue, Hardy, BMO) is discussed. One our main tools an appropriate version Hardy–Littlewood–Paley inequality ‖fˆ‖Lp′,q≲‖f‖Lp,q.

In this paper, some new nonlinear generalized Gronwall-Bellman-Type integral inequalities with mixed time delays are established. These inequalities can be used as handy tools to research stability problems of delayed differential and integral dynamic systems. As applications, based on these new established inequalities, some p-stable results of a integro-differential equation are also given. T...

Wavelets, which have many good properties such as time/freqency localization and compact support, are considered for solving the Hamilton-Jacobi-Bellman (HJB) equation as appears in optimal control problems. Specifically, we propose a Successive Wavelet Collocation Algorithm (SWCA) that uses interpolating wavelets in a collocation scheme to iteratively solve the Generalized-Hamilton-Jacobi-Bell...

We establish a connection between optimizing the Bellman Residual and worst case long-term predictive error. In the online learning framework, learning takes place over a sequence of trials with the goal of predicting a future discounted sum of rewards. Our analysis shows that, together with a stability assumption, any no-regret online learning algorithm that minimizes Bellman error ensures sma...

Standard Reinforcement Learning (RL) aims to optimize decision-making rules in terms of the expected return. However, especially for risk-management purposes, other criteria such as the expected shortfall are sometimes preferred. Here, we describe a method of approximating the distribution of returns, which allows us to derive various kinds of information about the returns. We first show that t...

t=1 γ (P 1P o2 . . . P o)t (Y |x) for all Y ⊆ X and x ∈ X. We will assume throughout this supplementary material that when we refer to an optimal policy π∗, it is a policy over primitive actions. Because we have assume that O contains the set of primitive actions A, the fixed point of the SMDP Bellman operator T and the MDP Bellman operator T is the optimal value function V ∗. Thus Tπ is equiva...