Decentralized Control of Stochastic Dynamic Systems with Applications to Resource Allocation and Portfolio Management by

Decentralized Control of Stochastic Dynamic Systems with Applications to Resource Allocation and Portfolio Management by Huaning Cai Doctor of Philosophy in Engineering Industrial Engineering & Operations Research University of California, Berkeley Professor Andrew E.B. Lim, Chair Modern engineering and social systems are often too complex to be managed by a centralized agent. Instead, such systems are commonly structured with multiple decentralized agents each responsible for managing a subset of the system, but the resulting system performance depends on the aggregate of the decisions made by decentralized agents. Local agents’ decision makings often exhibit selfish behavior as they seek to optimize their own objectives under their localized models, which if left uncoordinated can lead to substantial loss of efficiency compared with the system that can be optimized by a single (hypothetical) centralized agent. In this dissertation, we seek to study the fundamental issues of how to efficiently manage large-scale and multi-agent stochastic dynamic systems, especially on how to device efficient coordination mechanisms that would optimize system performance under various constraints that are unique to decentralized systems. In the first part of this dissertation we study decentralized control of a general class of stochastic dynamic resource allocation problems that have many applications. We consider a stochastic system in which multiple decentralized agents allocate shared system resources in response to customer requests that arrive stochastically over time. Each agent is responsible for a subset of the allocation decisions which it makes according to a dynamic allocation policy obtained by maximizing his own expected profit subject to a potentially mis-specified model of the way in which shared resources are consumed by other agents. We introduce the notion of a transfer contract which specifies how agents compensate one another whenever resources are consumed and establish the existence of contracts under which the decentralized system has no efficiency loss relative to centralized optimality. We also show that this property is insensitive to mis-specification by each agent of the dynamics of resource consumption by others in the system. An explicit characterization of the optimal transfer contract and an iterative decentralized algorithm for computing it is also provided. In the language of duality, contracts are analogous to shadow prices and the iterative algorithm has the favor of a dual update method, but strong duality and convergence of the it-