We consider the optimal control of an unreliable manufacturing system with restarting costs. In 1986 and 1988, Akella and Kumar (for the in nite horizon discounted cost) and Bielecki and Kumar (for the in nite horizon average expected cost) show that the optimal policy is given by an optimal inventory level (“hedging point policy”). Inspired by these simple systems, we explore a new class of mo...

This article discusses a reduction of discounted Continuous-Time Markov Decision Processes (CTMDPs) to discrete-time Markov Decision Processes (MDPs). This reduction is based on the equivalence of a randomized policy that chooses actions only at jump epochs to a nonrandomized policy that can switch actions between jumps. For discounted CTMDPs with bounded jump rates, this reduction was introduc...

Representation and modeling of economic uncertainty is addressed by different modeling methods, namely stochastic variables and probabilities, interval analysis, and fuzzy numbers, in particular triple estimates. Fo-cusing on discounted cash flow analysis numerical results are presented, comparisons are made between alter-native modeling methods, and characteristics of the methods are discussed.

We present a solution to some discounted optimal stopping problem for the maximum of a geometric Brownian motion on a finite time interval. The method of proof is based on reducing the initial optimal stopping problem with the continuation region determined by an increasing continuous boundary surface to a parabolic free-boundary problem. Using the change-of-variable formula with local time on ...

It has been more than ninety years since the classical square-root EOQ formula was given by Harris (1913). Yet there is no continuous-review stochastic inventory model published in the literature with a general enough demand, whose optimal policy would reduce to the square-root formula in the absence of the stochastic components of the underlying demand. Why? In this paper, we surmise the reaso...

We introduce an algebraic operator framework to study discounted penalty functions in renewal risk models. For inter-arrival and claim size distributions with rational Laplace transform, the usual integral equation is transformed into a boundary value problem, which is solved by symbolic techniques. The factorization of the differential operator can be lifted to the level of boundary value prob...

Abstract. Assuming that the forward rates f t are semimartingales, we give conditions on their components for the discounted bond prices to be martingales. To achieve this we give sufficient conditions for the integrated processes f̄ t = ∫ u 0 f t dv to be semimartingales and identify their various components. We recover the no-arbitrage conditions in well-known models in the literature, and fin...

In many applications of adaptive control it may be clear if the unknown parameters are constants or are time varying functions. It may be especially difficult to determine if some of the parameters are constants or slowly varying functions of time. To determine parameter variations it is necessary to "forget" the past of the state. The approach of exponential discounting of past information has...

An equivalent martingale measure selection strategy for discrete time, continuous state, asset price evolution models is proposed. The minimal martingale law is shown to generally fail to produce a probability law in this context. The proposed strategy, termed the Girsanov principle, performs a multiplicative decomposition of asset price movements into a predictable and martingale component wit...

We introduce the concept of a Markov risk measure and we use it to formulate risk-averse control problems for two Markov decision models: a finite horizon model and a discounted infinite horizon model. For both models we derive risk-averse dynamic programming equations and a value iteration method. For the infinite horizon problem we also develop a risk-averse policy iteration method and we pro...