Class Notes: Monte Carlo Methods Week 1, Direct Sampling

Monte Carlo means using random numbers to compute something that itself is not random. For example, suppose X is a random variable with some distribution, V (x) is some function, and we want to know A = E[V (X)]. There may be another random variable Y with another distribution, and another function W (y), so that A = E[W (Y )]. Simulation is the process of generating random variables with the X distribution for their own sakes. Much of the effort in Monte Carlo goes into looking for alternative ways to estimate the same number. The alternatives may have lower variance or be easier to evaluate. This is a difference between Monte Carlo and simulation. There are many scientific problems for which Monte Carlo is among the best known solution methods. Most of these arise through the curse of dimensionality. In its simplest form, this curse is that it is impractical to create a mesh for numerical integration or for PDE solving in high dimensions. A mesh with n points on a side in d dimensions has n mesh points in total. This is impractical, for example, if n = 10 and d = 50. A more general version of the curse is that it is impossible to represent a generic function f(x) in high dimensions. Consider a general polynomial in d variables. A polynomial is a linear combination of monomials. The number of monomials x1 1 · · ·x kd d of degree ≤ n is ( n+d n ) . This is the number of coefficients you need to represent a general polynomial of degree n. For example, you need about 10, 000 coefficients for degree 4 in 20 variables, and about thirty million coefficients for a degree 10 polynomial in 20 variables. For example, dynamic programming is an algorithm that requires you to represent the “value function” (for economists) or the “cost to go function” (for engineers). Dynamic programming is impractical except for low dimensional problems. On the other hand, if f(x) is a probability density, it may be possible to represent f using a large number of samples. A sample of f is a random variable X that has f as its probability density. If Xk for k = 1, ..., N is a collection of samples, then