Separating Mixtures Using Megapriorstimothy

Fitting the parameters of a discrete nite mixture distribution to a set of data using the EM algorithm can be extremely diicult when the likelihood surface has many local maxima, the form of the components is unknown, and the number of components is unknown. The exponential explosion in the number of diierent models and diierent starting points for EM which must be tested can be reduced by nding the individual components of the mixture distribution one-at-a-time. This can be done by tting a succession of two-component models to the data. Each two-component model has one component constrained to be the uniform distribution and, ideally, when EM has converged, the other component of the model is a single component of the sampled distribution. Unfortunately, the second component of the tted model is often a convex combination of two or more components of the distribution. This diiculty can be overcome by using an extremely low variance prior on the parameters of the components of the distribution and maximizing the posterior probability of the data rather than its likelihood using a modiied version of EM. In particular, a \megaprior"|a prior whose variance is a function of the sample size| insures that the convex combination problem is avoided no matter how large the sample. We have incorporated these ideas into our MEME algorithm for discovering patterns (motifs) in biological sequences. Experimentally determined priors are available for the parameters of protein motifs. We decrease the variance of these priors to further reduce the algorithm's tendency to nd convex combinations of motifs. Experiments with protein sequence data show that by using megapriors the problem of convex combinations is avoided and our algorithm discovers biologically signiicant patterns.