This paper addresses the estimation of parameters of a Bayesian network from incomplete data. The task is usually tackled by running the Expectation-Maximization (EM) algorithm several times in order to obtain a high log-likelihood estimate. We argue that choosing the maximum log-likelihood estimate (as well as the maximum penalized log-likelihood and the maximum a posteriori estimate) has seve...

Many problems in natural language processing can be viewed as lin guistic classi cation problems in which linguistic contexts are used to pre dict linguistic classes Maximum entropy models o er a clean way to com bine diverse pieces of contextual evidence in order to estimate the proba bility of a certain linguistic class occurring with a certain linguistic con text This report demonstrates the...

We consider the problem of estimating Shannon’s entropy H from discrete data, in cases where the number of possible symbols is unknown or even countably infinite. The Pitman-Yor process, a generalization of Dirichlet process, provides a tractable prior distribution over the space of countably infinite discrete distributions, and has found major applications in Bayesian non-parametric statistics...

Entropy is a measure of probabilistic uncertainty. In 1957, E.T. Jaynes introduced his idea of the principle of Maximum Entropy. This idea states that by finding a distribution with the maximum entropy, one maximizes the uncertainty constrained to the given information. In essence, this involves using all of the known information, and assuming nothing about the distribution. In this report, we ...

Efficient estimation of the moments and Shannon entropy of data streams is an important task in modern machine learning and data mining. To estimate the Shannon entropy, it suffices to accurately estimate the α-th moment with ∆ = |1 − α| ≈ 0. To guarantee that the error of estimated Shannon entropy is within a ν-additive factor, the method of symmetric stable random projections requires O ( 1 ν...

We carry out a comprehensive analysis of letter frequencies in contemporary written Marathi. We determine sets of letters which statistically predominate any large generic Marathi text, and use these sets to estimate the entropy of Marathi.

We previously introduced the SegSLAM algorithm, an approach to the simultaneous localization and mapping (SLAM) problem that divides the environment up into segments, or submaps, using heuristic methods. We investigate a realtime method for Active SLAM with SegSLAM, in which actions are selected in order to reduce uncertainty in both the local metric submap and the global topological map. Recen...

Graduation of data is of great importance in survival analysis. Smoothness and goodness of fit are two fundamental requirements in graduation. Based on the instinctive defining expression for entropy in terms of a probability distribution, two optimization models based on the Maximum Entropy Principle (MaxEnt) and Minimum Cross Entropy Principle (MinCEnt) to estimate mortality probability distr...

The Wireless Multimedia Sensor Network consists cameras at the sensor node in visual data application. These camera sensor devices capture their observations limited by the FoV as an image. There is a correlation between images captured by multiple cameras at a particular area. This leads to the redundant data transmission in the network. As the sensor node are battery powered and resource limi...