The expectation maximisation (EM) algorithm has proven to be e ective for a range of identi cation problems. Unfortunately, the way in which the EM algorithm has previously been applied has proven unsuitable for the commonly employed innovations form model structure. This paper addresses this problem, and presents a previously unexamined method of EM algorithm employment. The results are pro le...

The EM (expectation-maximization) algorithm is a broadly applicable method for calculating maximum likelihood estimates given incomplete data [1]. EM algorithms have received considerable attention due to their computation feasibility in tomographic image reconstruction [2~4], symbol detection [5] and parameter estimation [6]. However, it is less recognized that EM algorithms can be equally app...

A robust family of algorithms generalizing the EM-algorithm for fitting parametric deterministic multi-trajectories observed in Gaussian noise and clutter is proposed. It is based on the M-estimation generalizing the Maximum Likelihood estimation in the M-step of the EM-algorithm. Simulation results of comparative performance of our and traditional EM-algorithm in noise and clutter are described.

Estimating the mean and the covariance matrix of an incomplete dataset and filling in missing values with imputed values is generally a nonlinear problem, which must be solved iteratively. The expectation maximization (EM) algorithm for Gaussian data, an iterative method both for the estimation of mean values and covariance matrices from incomplete datasets and for the imputation of missing val...

The celebrated expectation-maximization (EM) algorithm is one of the most widely used optimization methods in statistics. In recent years it has been realized that EM algorithm is a special case of the more general minorization-maximization (MM) principle. Both algorithms creates a surrogate function in the first (E or M) step that is maximized in the second M step. This two step process always...

This paper introduces variational expectation-maximization (VEM) algorithm for training Gaussian networks. Hyperparameters model distributions of parameters characterizing Gaussian mixture densities. The proposed algorithm employs a hierarchical learning strategy for estimating a set of hyperparameters and the number of Gaussian mixture components. A dual EM algorithm is employed as the initial...

3 Clustering algorithms 5 3.1 ML/MAP based clustering . . . . . . . . . . . . . . 5 3.1.1 Parameter estimation based on ML . . . . . . 5 3.1.2 Parameter estimation based on MAP . . . . . 5 3.1.3 Membership distribution . . . . . . . . . . . 6 3.1.4 Dissimilarity . . . . . . . . . . . . . . . . . 7 3.1.5 Clustering . . . . . . . . . . . . . . . . . . . 7 3.1.6 Relevance analysis . . . . . . . . ...

Model-based clustering for functional data is considered. An alternative to model-based clustering using the functional principal components is proposed by approximating the density of functional random variables. An EM-like algorithm is used for parameter estimation and the maximum a posteriori rule provides the clusters. Real data applications illustrate the interest of the proposed methodology.

In the previous class we already mentioned that many of the most powerful probabilistic models contain hidden variables. We will denote these variables with y. It is usually also the case that these models are most easily written in terms of their joint density, p(d,y,θ) = p(d|y,θ) p(y|θ) p(θ) (1) Remember also that the objective function we want to maximize is the log-likelihood (possibly incl...

The EM algorithm is a very powerful optimization method and has reached popularity in many fields. Unfortunately, EM is only a local optimization method and can get stuck in suboptimal solutions. While more and more contemporary data/model combinations yield more than one optimum, there have been only very few attempts at making EM suitable for global optimization. In this paper we review the b...