We introduce a novel approach named unambiguity regularization for unsupervised learning of probabilistic natural language grammars. The approach is based on the observation that natural language is remarkably unambiguous in the sense that only a tiny portion of the large number of possible parses of a natural language sentence are syntactically valid. We incorporate an inductive bias into gram...

This paper is about the estimation of fixed model parameters in hidden Markov models using an online (or recursive) version of the Expectation-Maximization (EM) algorithm. It is first shown that under suitable mixing assumptions, the large sample behavior of the traditional (batch) EM algorithm may be analyzed through the notion of a limiting EM recursion, which is deterministic. This observati...

Maximum Likelihood estimation based Expectation Maximization(EM) reconstruction algorithm [ 11 has been shown to provide good quality reconstruction for PET. Our previous work [2,3] introduced multigrid concept for PET image reconstruction using EM. The multiresolution EM (MREM) algorithm is an attempt to improve the EM based estimation through an effective use of multi-resolution grids in both...

Electron velocity distribution obtained by direct spacecraft observation in space is contaminated by photoelectrons. The photoelectrons are generated due to the solar ultraviolet ray, and are regarded as artificial noise from a viewpoint of scientific research. We propose a method for separating photoelectron component from ambient electron component. Our method uses multivariate normal mixture...

A basic approach to estimation of mixture model parameters is by using expectation maximization (EM) algorithm for maximizing the likelihood function. However, it is essential to provide the algorithm with proper initial conditions, as it is highly dependent on the first estimation (“guess”) of parameters of a mixture. This paper presents several different initial condition estimation methods, ...

In this report, we develop a procedure to analyze the relationship between the observed multi-dimensional counts and a set of explanatory variables. The counts follow a multivariate Poisson distribution or a multivariate zero-inflated Poisson distribution. Maximum likelihood estimates (MLE) for the model parameters are obtained by the Newton-Raphson (NR) iteration and the expectation-maximizati...

In this paper an iterative method for semi-blind MIMO channel identification and tracking is presented. The method is based on results from information geometry; specifically, the alternating projections theorem first proved by Csiszar [2], which provides a rigorous iterative method for stochastic maximum likelihood estimation. It is demonstrated that the proposed method has similar performance...

We show a close relationship between the Expectation Maximization (EM) algorithm and direct optimization algorithms such as gradientbased methods for parameter learning. We identify analytic conditions under which EM exhibits Newton-like behavior, and conditions under which it possesses poor, first-order convergence. Based on this analysis, we propose two novel algorithms for maximum likelihood...

We propose a novel approach to estimate the angles of arrival AOA and delays of the multipath signals from digitally modulated sources ar riving at an antenna array Our method uses a collection of estimates of a space time vector channel The method avoids computationally expensive optimization search by using Expectation Maximization EM algorithm The useful behavior of the proposed algorithm is...