We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along time horizon generates an overall solution by simply concatenating optimal solutions from these sub-problems. As “black box” architecture, works with wide range of existing algorithms. Our main results characteriz...

We consider the problem of designing policies for Markov decision processes (MDPs) with dynamic coherent risk objectives and constraints. begin by formulating in a Lagrangian framework. Under assumption that constraints can be represented transition mapping, we propose an optimization-based method to synthesize Markovian lower-bound constrained risk-averse problem. demonstrate formulated optimi...

We consider the framework of transfer-entropy-regularized Markov decision process (TERMDP) in which weighted sum classical state-dependent cost and transfer entropy from state random to control input is minimized. Although TERMDPs are generally formulated as nonconvex optimization problems, an analytical necessary optimality condition can be expressed a finite set nonlinear equations, based on ...