Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward ...

• We often approximate a large number of periods, even if the horizon is known and finite, by assuming an infinite number of periods, and hope that this assumption will simplify the solution. Indeed, even if the general theory becomes more involved, the solution obtained often is simpler and has important computational and conceptual advantages: in particular, the optimal policy is often statio...

This is the first of the two companion papers which treat an infinite time horizon hereditary portfolio optimization problem in a market that consists of one savings account and one stock account.Within the solvency region, the investor is allowed to consume from the savings account and can make transactions between the two assets subject to paying capital gain taxes as well as a fixed plus pro...

This paper presents an infinite-dimensional Receding Horizon Observer for linear 2x2 hyperbolic systems with boundary measurements. The initial state is estimated as the optimal solution of an optimization problem which minimizes the distance between the measurements and the observer output. A constructive method is used to derive the existence and uniqueness of the solution. A composite strate...

We study the possible existence of black holes in scalar-tensor theories of gravity in four dimensions. Their existence is verified for anomalous versions of these theories, with a negative kinetic term in the Lagrangian. The Hawking temperature TH of these holes is zero, while the horizon area is (in most cases) infinite. It is shown that an infinite value of TH can occur only at a curvature s...

In this paper, we present new RHNHC (Receding Horizon Neural H∞ Control) for nonlinear unknown systems. First, we propose LMI (Linear Matrix Inequality) condition on the terminal weighting matrix for stabilizing RHNHC. Under this condition, noninceasing monotonicity of the saddle point value of the finite horizon dynamic game is shown to be guaranteed. Then, we propose RHNHC for nonlinear unkno...

We consider a periodic review inventory system and present its optimal policy in the infinite horizon setting. The optimal inventory policy that maximizes the infinite horizon expected discounted profit for the model is analytically obtained by relating to the finite horizon setting using results from variational analysis. Results are provided that elucidate the operations of the inventory syst...

An important aspect of numerically approximating the solution of an infinite-horizon optimal control problem is the manner in which the horizon is treated. Generally, an infinite-horizon optimal control problem is approximated with a finite-horizon problem. In such cases, regardless of the finite duration of the approximation, the final time lies an infinite duration from the actual horizon at ...

We prove existence of Markov Perfect Equilibria (MPE) in nonstationary undiscounted infinite horizon dynamic games, by exploiting a structural property (Uniformly Bounded Reachability) of the state dynamics. This allows us to identify a suitable finite horizon equilibrium relaxation, the ending state Constrained MPE, that captures the relevant features of an infinite horizon MPE for a long enou...