This paper deals with the stochastic control of nonlinear systems in the presence of state and control constraints, for uncertain discrete-time dynamics in finite dimensional spaces. In the deterministic case, the viability kernel is known to play a basic role for the analysis of such problems and the design of viable control feedbacks. In the present paper, we show how a stochastic viability k...

We present a general aggregation method applicable to all finite-horizon Markov decision problems. States of the MDP are aggregated into macro-states based on a pre-selected collection of “distinguished” states which serve as entry points into macro-states. The resulting macro-problem is also an MDP, whose solution approximates an optimal solution to the original problem. The aggregation scheme...

This note presents some historical examples ofthe link between the main areas of mathematics and thestatistical theory. The research in statistics has an impacton algebra and analysis as much as the innovations dueto the probability theory, while algebra and analysisimprove the statistical methods.

We present experimental results about learning function values (i.e. Bellman values) in stochastic dynamic programming (SDP). All results come from openDP (opendp.sourceforge.net), a freely available source code, and therefore can be reproduced. The goal is an independent comparison of learning methods in the framework of SDP. 1 What is stochastic dynamic programming (SDP) ? We here very roughl...

We investigate in this paper submodular value functions using complex dynamic programming. In complex dynamic programming (dp) we consider concatenations and linear combinations of standard dp operators, as well as combinations of maximizations and minimizations. These value functions have many applications and interpretations, both in stochastic control (and stochastic zero-sum games) as well ...

Multistage stochastic integer programming (MSIP) combines the difficulty of uncertainty, dynamics, and non-convexity, and constitutes a class of extremely challenging problems. A common formulation for these problems is a dynamic programming formulation involving nested cost-to-go functions. In the linear setting, the cost-to-go functions are convex polyhedral, and decomposition algorithms, suc...

1. Introduction Many important real-world problems contain stochastic elements and require optimization. Stochastic programming and simulation-based optimization are two approaches used to address this issue. We do not explicitly discuss other related areas including stochastic control, stochas-tic dynamic programming, and Markov decision processes. We consider a stochastic optimization problem...

Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are regained, and solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimizat...

The purpose of this paper is to present a framework for optimal stochastic power control in interference limited fading wireless channels. The framework is based on applying stochastic programming methods to optimal transmit power allocation. Numerical examples based on applying interior point algorithms to the optimal power control problem are discussed. Furthermore, an application of the stoc...