The EM algorithm is not a single algorithm, but a template for the construction of iterative algorithms. While it is always presented in stochastic language, relying on conditional expectations to obtain a method for estimating parameters in statistics, the essence of the EM algorithm is not stochastic. The conventional formulation of the EM algorithm given in many texts and papers on the subje...

This paper investigates the effectiveness of the DAEM (Deterministic Annealing EM) algorithm in acoustic modeling for speaker and speech recognition. Although the EM algorithm has been widely used to approximate the ML estimates, it has the problem of initialization dependence. To relax this problem, the DAEM algorithm has been proposed and confirmed the effectiveness in artificial small tasks....

The present work proposes hybridization of Expectation-Maximization (EM) and KMeans techniques as an attempt to speed-up the clustering process. Though both K-Means and EM techniques look into different areas, K-means can be viewed as an approximate way to obtain maximum likelihood estimates for the means. Along with the proposed algorithm for hybridization, the present work also experiments wi...

In this paper we present new results relative to the "expectation-maximization/maximization of the posterior marginals" (EM/MPM) algorithm for simultaneous parameter estimation and segmentation of textured images. The EM/MPM algorithm uses a Markov random field model for the pixel class labels and alternately approximates the MPM estimate of the pixel class labels and estimates parameters of th...

The EM algorithm of Dempster, Laird, and Rubin (1977) is a broadly applicable approach that has been widely applied to the iterative computation of maximum likelihood estimates in a variety of incomplete-data problems. A criticism that has been levelled at the EM algorithm is that its convergence can be quite slow. Unfortunately, methods to accelerate the EM algorithm do tend to sacriice the si...

The Expectation-Maximization (EM) algorithm is a general technique for maximum likelihood estimation (MLE). In this paper we present several of the important theoretical and practical issues associated with Gaussian mixture mod-eling (GMM) within the EM framework. First, we propose an EM algorithm for estimating the parameters of a special GMM structure, named a probablistic mapping network (PM...

y In this paper, a space-alternating generalized expectation-maximization (SAGE) algorithm is presented for the numerical computation of maximum-likelihood (ML) and penalized maximum-likelihood (PML) estimates of the parameters of covariance matrices with linear structure for complex Gaussian processes. By using a less informative hidden-data space and a sequential parameter-update scheme, a SA...

This EM review article focuses on parameter expansion, a simple technique introduced in the PX-EM algorithm to make EM converge faster while maintaining its simplicity and stability. The primary objective concerns the connection between parameter expansion and efficient inference. It reviews the statistical interpretation of the PX-EM algorithm, in terms of efficient inference via bias reductio...

The expectation-maximization (EM) algorithm for maximum-likelihood image recovery is guaranteed to converge, but it converges slowly. Its ordered-subset version (OS-EM) is used widely in tomographic image reconstruction because of its order-of-magnitude acceleration compared with the EM algorithm, but it does not guarantee convergence. Recently the ordered-subset, separable-paraboloidal-surroga...

Finite mixture models implemented via the EM algorithm are being increasingly used in a wide range of problems in the context of unsupervised statistical pattern recognition. As each E-step visits each feature vector on a given iteration, the EM algorithm requires considerable computation time in its application to large data sets. We consider two approaches, an incremental EM (IEM) algorithm a...