Between sensing the world after every action (as in a reactive plan) and not sensing at all (as in an openloop plan), lies a continuum of strategies for sensing during plan execution. If sensing incurs a cost (in time or resources), the most cost-effective strategy is likely to fall somewhere between these two extremes. Yet most work on plan execution assumes one or the other. In this paper, an...

multistage stochastic programming is a key technology for making decisions over time in an uncertain environment. one of the promising areas in which this technology is implementable, is medium term planning of electricity production and trading where decision makers are typically faced with uncertain parameters (such as future demands and market prices) that can be described by stochastic proc...

Dynamic probabilistic networks are a compact representation of complex stochastic processes. In this paper we examine how to learn the structure of a DPN from data. We extend structure scoring rules for standard probabilistic networks to the dynamic case, and show how to search for structure when some of the variables are hidden. Finally, we examine two applications where such a technology migh...

wind power generation is variable and uncertain. in the power systems with high penetration of wind power, determination of equivalent operating reserve is the main concern of systems operator. in this paper, a model is proposed to determine operating reserves in simultaneous market clearing of energy and reserve by stochastic programming based on scenarios generated via monte carlo simulation ...

We consider risk-averse formulations of multistage stochastic linear programs. For these formulations, based on convex combinations of spectral risk measures, risk-averse dynamic programming equations can be written. As a result, the Stochastic Dual Dynamic Programming (SDDP) algorithm can be used to obtain approximations of the corresponding risk-averse recourse functions. This allows us to de...

In this paper we introduce and solve the partially observed optimal stopping nonlinear risk-sensitive stochastic control problem for discrete-time non-linear systems. The presented results are closely related to previous results for finite horizon partially observed risk-sensitive stochastic control problem. An information state approach is used and a new (three-way) separation principle establ...

Aim of this paper is to set invariance in stochastic dynamical control systems. Given a set within which the state of the dynamical system should evolve, we study conditions for finding a control strategy that maximizes the probability for the state to be in the given set within a fixed a–priori finite time horizon. We formulate an optimal control problem and we solve the problem at hand by usi...

This paper describes a stochastic dynamic programming based approach to solve Sensor Resource Management (SRM) problems such as occur in tracking multiple targets with electronically scanned, multi-mode radar. Specifically, it formulates the SRM problem as a stochastic scheduling problem and develops approximate solutions based on the Gittins index rule. Novel results include a hybrid state sto...

Motivated by applications in mathematical finance [3] and stochastic analysis [16], we continue our study of second order backward stochastic equations (2BSDE). In this paper, we derive the dynamic programming equation for a certain class of problems which we call as the second order stochastic target problems. In contrast with previous formulations of similar problems, we restrict control proc...