This paper presents a class of new approximate policy iteration algorithms for solving infinite-horizon, discounted Markov decision processes (MDPs) for which a model of the system is available. The algorithms are similar in spirit to Bellman residual minimization methods. However, by exploiting kernel-based regression techniques with nondegenerate kernel functions as the underlying cost-to-go ...

With the development of connected and automated vehicles, eco-driving control is reckoned to generate unprecedented potential on energy-saving in electrified powertrain. In this paper, a data-driven based strategy with efficient computation capacity proposed for plug-in hybrid electric vehicles achieve approximate optimal energy economy. An hierarchical scheme designed mitigate massive computat...

In this paper, we study a class of zero-sum two-player stochastic differential games with the controlled equations and payoff/cost functionals recursive type. As opposed to pioneering work by Fleming Souganidis [Indiana Univ. Math. J. 38 (1989) 293–314] seminal Buckdahn Li [SIAM Control Optim. 47 (2008) 444–475], involved coefficients may be random, going beyond Markovian framework leading rand...

We study utility maximization for power utility random elds with and without intermediate consumption in a general semimartingale model with closed portfolio constraints. We show that any optimal strategy leads to a solution of the corresponding Bellman equation. The optimal strategies are described pointwise in terms of the opportunity process, which is characterized as the minimal solution of...

Evolutionary programming is a stochastic optimization procedure which has proved useful in optimizing difficult functions. It is shown that evolutionary programing can be used to solve the Bellman equation problem with a high degree of accuracy and substantially less CPU time than Bellman equation iteration. Future applications will focus on sometimes binding constraints – a class of problem fo...

We obtain Fourier inequalities in the weighted Lp spaces for any 1<p<∞ involving Hardy–Cesàro and Hardy–Bellman operators. extend these results to product Hardy p⩽1. Moreover, boundedness of Hardy-Cesàro Hardy-Bellman operators various (Lebesgue, Hardy, BMO) is discussed. One our main tools an appropriate version Hardy–Littlewood–Paley inequality ‖fˆ‖Lp′,q≲‖f‖Lp,q.

In this paper, some new nonlinear generalized Gronwall-Bellman-Type integral inequalities with mixed time delays are established. These inequalities can be used as handy tools to research stability problems of delayed differential and integral dynamic systems. As applications, based on these new established inequalities, some p-stable results of a integro-differential equation are also given. T...

Wavelets, which have many good properties such as time/freqency localization and compact support, are considered for solving the Hamilton-Jacobi-Bellman (HJB) equation as appears in optimal control problems. Specifically, we propose a Successive Wavelet Collocation Algorithm (SWCA) that uses interpolating wavelets in a collocation scheme to iteratively solve the Generalized-Hamilton-Jacobi-Bell...

We establish a connection between optimizing the Bellman Residual and worst case long-term predictive error. In the online learning framework, learning takes place over a sequence of trials with the goal of predicting a future discounted sum of rewards. Our analysis shows that, together with a stability assumption, any no-regret online learning algorithm that minimizes Bellman error ensures sma...

Standard Reinforcement Learning (RL) aims to optimize decision-making rules in terms of the expected return. However, especially for risk-management purposes, other criteria such as the expected shortfall are sometimes preferred. Here, we describe a method of approximating the distribution of returns, which allows us to derive various kinds of information about the returns. We first show that t...