A Tutorial on the Dirichlet Process for Engineers Technical Report

This document provides a review of the Dirichlet process originally given in the author’s preliminary exam paper and is presented here as a tutorial. No motivation is given (in what I’ve excerpted here), and this document is intended to be a mathematical tutorial that is still accessible to the engineer. I. THE DIRICHLET DISTRIBUTION Consider the finite, D-dimensional vector, π, having the properties 0 ≤ πi ≤ 1, i = 1, . . . , D and ∑D i=1 πi = 1 (i.e., residing on the (D − 1)-dimensional simplex in RD, or π ∈ ∆D). We view this vector as the parameter for the multinomial distribution, where samples, X ∼ Mult(π), take values X ∈ {1, . . . , D} with probability P (X = i|π) = πi. When the vector π is unknown, it can be inferred in the Bayesian setting using its conjugate prior, the Dirichlet distribution. The Dirichlet distribution of dimension D is a continuous probability measure on ∆D having the density function p(π|β1, . . . , βD) = Γ\( ∑ i βi) ∏ i Γ\(βi) D ∏