New Markov Decision Process Formulations and Optimal Policy Structure for Assemble-to-Order and New Product Development Problems

This thesis examines two complex, dynamic problems by employing the theory of Markov Decision Processes (MDPs). Chapters 2 and 3 consider assemble-to-order (ATO) inventory systems. An ATO system consists of several components and several products, and assembles products as demand is realized; it is becoming increasingly popular since it provides greater flexibility in manufacturing at a reasonable cost. This work contributes to the ATO research stream by characterizing optimal inventory replenishment and allocation policies. Chapter 4 examines the new product development (NPD) process with scarce resources and many projects in parallel, each lasting several periods, in the face of uncertainty. This study advances the NPD literature by revealing that optimal project selection and resource allocation decisions are congestion-dependent. Below, I elaborate on the novel optimal policies and structural results I obtain using MDP formulations, which is the overarching theme of the thesis. In Chapter 2, I consider generalized ATO “M -systems” with multiple components and multiple products. These systems involve a single “master” product which uses multiple units from each component, and multiple individual products each of which consumes multiple units from a different component. Such systems are common for manufacturers selling an assembled product as well as individual spare parts. I model these systems as infinite-horizon MDPs under the discounted cost criterion. Each component is produced in batches of fixed size in a make-to-stock fashion; batch sizes are determined by individual product sizes. Production times are independent and exponentially distributed. Demand for each product arrives as an independent Poisson process. If not satisfied immediately upon arrival, these demands are lost. Therefore the state of the system can be described by component inventory levels. A control policy specifies when a batch of components should be produced (i.e., inventory