We present a general framework containing a graded spectrum of Expectation Maximization (EM) algorithms called Unified Expectation Maximization (UEM.) UEM is parameterized by a single parameter and covers existing algorithms like standard EM and hard EM, constrained versions of EM such as ConstraintDriven Learning (Chang et al., 2007) and Posterior Regularization (Ganchev et al., 2010), along w...

This paper investigates the effectiveness of the DAEM (Deterministic Annealing EM) algorithm in acoustic modeling for speaker and speech recognition. Although the EM algorithm has been widely used to approximate the ML estimates, it has the problem of initialization dependence. To relax this problem, the DAEM algorithm has been proposed and confirmed the effectiveness in small tasks. In this pa...

We discuss the integration of the expectation-maximization (EM) algorithm for maximum likelihood learning of Bayesian networks with belief propagation algorithms for approximate inference. Specifically we propose to combine the outer-loop step of convergent belief propagation algorithms with the M-step of the EM algorithm. This then yields an approximate EM algorithm that is essentially still d...

We present a deterministic annealing variant of the EM algorithm for maximum likelihood parameter estimation problems. In our approach, the EM process is reformulated as the problem of minimizing the thermodynamic free energy by using the principle of maximum entropy and statistical mechanics analogy. Unlike simulated annealing approaches, this minimization is deterministically performed. Moreo...

In this paper, an improved expectation maximization (EM) algorithm called statistical histogram based expectation maximization (SHEM) algorithm is presented. The algorithm is put forward to overcome the drawback of standard EM algorithm, which is extremely computationally expensive for calculating the maximum likelihood (ML) parameters in the statistical segmentation. Combining the SHEM algorit...

In SPECT/PET, the maximum-likelihood expectation-maximization (ML-EM) algorithm is getting more attention as the speed of computers increases. This is because it can incorporate various physical aspects into the reconstruction process leading to a more accurate reconstruction than other analytical methods such as filtered-backprojection algorithms. However, the convergence rate of the ML-EM alg...

In this paper, the maximum likelihood estimation of a general two-level structural equation model with an unbalanced design is formulated as a missing data problem by treating the latent random vectors at the group level as hypothetical missing data. The commonly used EM algorithm is utilized to obtain the solution. Expressions for the E-step are derived and it is shown that the complex optimiz...

3 The Expectation-Maximization algorithm 7 3.1 Jointly-non-concave incomplete log-likelihood . . . . . . . . . . . 7 3.2 (Possibly) Concave complete data log-likelihood . . . . . . . . . . 8 3.3 The general EM derivation . . . . . . . . . . . . . . . . . . . . . 10 3.4 The E& M-steps . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.5 The EM algorithm . . . . . . . . . . . . . . . . . . ...

In this paper we propose algorithms for parameter estimation of fast-sampled homogeneous Markov chains observed in white Gaussian noise. Our algorithms are obtained by the robust discretization of stochastic differential equations involved in the estimation of continuous-time Hidden Markov Models (HMM’s) via the EM algorithm. We present two algorithms: The first is based on the robust discretiz...

EDML is a recently proposed algorithm for learning MAP parameters in Bayesian networks. In this paper, we present a number of new advances and insights on the EDML algorithm. First, we provide the multivalued extension of EDML, originally proposed for Bayesian networks over binary variables. Next, we identify a simplified characterization of EDML that further implies a simple fixed-point algori...