CSE 455/555 Spring 2013 Homework 7: Parametric Techniques

(a) Write down the probability distribution of x. Solution: p(x = 0|μ) = 1− μ, so p(x|μ) = μx(1− μ)1−x, this is known as the Bernoulli distribution. (b) Show that this is a proper probability distribution, i.e. the probability sum up to 1. What is the expectation and variance of this distribution? Solution: ∑ x∈{0,1} p(x|μ) = p(x = 0|μ) + p(x = 1|μ) = 1− μ+ μ = 1 E(x) = ∑ x∈{0,1} xp(x|μ) = 0× p(x = 0|μ) + 1× p(x = 1|μ) = μ var(x) = {E(x2)− E(x)2} = ∑ x∈{0,1} x 2p(x|μ)− μ2 = 02 × p(x = 0|μ) + 12 × p(x = 1|μ)− μ2 = μ− μ2 (c) Now suppose we have a set of observed values X = {x1, x2, . . . , xN} of x. Write the likelihood function and estimate the maximum likelihood parameter μML. Solution: p(X|μ) = ∏N i=1 p(xi|μ) = ∏N i=1 μ xi(1− μ)1−xi The log-likelihood can be written as: l(X|μ) = ∑N i=1 ln p(xi|μ) = ∑N i=1{xi lnμ+ (1− xi) ln(1− μ)} Take the derivative w.r.t. μ and set to zero we get: