A portfolio optimization problem on an infinite time horizon is considered. Risky asset price obeys a logarithmic Brownian motion, and the interest rate varies according to an ergodic Markov diffusion process. Moreover, the interest rate fluctuation is correlated with the risky asset price fluctuation. The goal is to choose optimal investment and consumption policies to maximize the infinite ho...

We develop a general theory of efficient policy gradient algorithms for Noise-Action MDPs (NMDPs), a class of MDPs that generalize Linearly Solvable MDPs (LMDPs). For finite horizon problems, these lead to simple update equations based on multiple rollouts of the system. We show that our policy gradient algorithms are faster than the PI algorithm, a state of the art policy optimization algorith...

We discuss a class of risk-sensitive portfolio optimization problems. We consider the portfolio optimization model investigated by Nagai in 2003. The model by its nature can include fixed income securities as well in the portfolio. Under fairly general conditions, we prove the existence of optimal portfolio in both finite and infinite horizon problems.

We present results on numerical regulator design for sampled-data nonlinear plants via their approximate discrete-time plant models. The regulator design is based on an approximate discrete-time plant model and is carried out either via an infinite horizon optimization problem or via a finite horizon with terminal cost optimization problem. In both cases we discuss situations when the sampling ...

We present results on numerical regulator design for sampled-data nonlinear plants via their approximate discrete-time plant models. The regulator design is based on an approximate discrete-time plant model and is carried out either via an infinite horizon optimization problem or via a finite horizon with terminal cost optimization problem. We focus on the case when the sampling period T and th...

Condition-Based Maintenance (CBM) optimization using Proportional Hazards Model (PHM) is a kind of maintenance optimization problem in which inspections of a system relevant to its failure rate depending on the age and value of covariates are performed in time intervals. The general approach for constructing a CBM based on PHM for a system is to minimize a long run average cost per unit of time...

We consider the problem of selecting an optimality criterion, when total costs diverge, in deterministic infinite horizon optimization over discrete time. Our formulation allows for both discrete and continuous state and action spaces, as well as time-varying, that is, nonstationary, data. The task is to choose a criterion that is neither too overselective, so that no policy is optimal, nor too...

Decentralized POMDPs provide an expressive framework for multi-agent sequential decision making. While finite-horizon DECPOMDPs have enjoyed significant success, progress remains slow for the infinite-horizon case mainly due to the inherent complexity of optimizing stochastic controllers representing agent policies. We present a promising new class of algorithms for the infinite-horizon case, w...