Sample Average Approximation Method for Compound Stochastic Optimization Problems

The paper studies stochastic optimization (programming) problems with compound functions containing expectations and extreme values of other random functions as arguments. Compound functions arise in various applications. A typical example is a variance function of nonlinear outcomes. Other examples include stochastic minimax problems, econometric models with latent variables, multi-level and multicriteria stochastic optimization problems. As a solution technique a Sample Average Approximation (SAA) method (also known as statistical or empirical (sample) mean method) is used. The method consists in approximation of all expectation functions by their empirical means and solving the resulting approximate deterministic optimization problems. In stochastic optimization this method is widely used for optimization of standard expectation functions under constraints. In this paper SAA method is extended to general compound stochastic optimization problems. The conditions for convergence in mean, almost surely and rate of convergence are established. The study of the convergence rate is based on properties of Rademacher averages of functional sets, concentration inequalities for bounded random functions and on the concept of uniform normalized convergence of random variables. The convergence results are applicable both for discrete and continuous stochastic optimization problems.