Clustering is an unsupervised process to determine which unlabeled objects in a set share interesting properties. The objects are grouped into k subsets (clusters) whose elements optimize a proximity measure. Methods based on information theory have proven to be feasible alternatives. They are based on the assumption that a cluster is one subset with the minimal possible degree of “disorder”. T...

In this paper we address the problem of building convenient criteria to solve linear and noisy inverse problems of the form y = Ax + n. Our approach is based on the speciication of constraints on the solution x through its belonging to a given convex set C. The solution is chosen as the mean of the distribution which is the closest to a reference measure on C with respect to the Kullback diverg...

— Following the ideas of Bontekoe et al. who noticed that the classical Maximum Entropy Method (MEM) had difficulties to efficiently restore high and low spatial frequency structure in an image at the same time, we use the wavelet transform, a mathematical tool to decompose a signal into different frequency bands. We introduce the concept of multi-scale entropy of an image, leading to a better ...

The maximum entropy method (MEM) is a relatively new technique for solving underdetermined systems. It has been successfully applied in many different areas. All methods for solving underdetermined systems introduce some additional, artificial constraints. The advantage of the maximum entropy method is that it uses the most natural additional constraint: one that does not introduce any new, arb...

In this paper, we present a combined approach that automatically extracts opinions from Arabic documents. Most research efforts in the area of opinion mining deal with English texts and little work with Arabic text. Unlike English, from our experiments, we found that using only one method on Arabic opinioned documents produce a poor performance. So, we used a combined approach that consists of ...

The paper proposes a sentence alignment method based on maximum entropy model using anchor sentences to align ancient and modern Chinese sentences in historical classics. The method selects the sentence pairs with the same phrases at the beginning or the end of the sentence or with the same time phrases as anchor sentence pairs, which are employed to divide the paragraph into several sections. ...

In this paper, we have developed conditions under which the entropy function and the residual entropy function characterize the distribution. We have also studied some stochastic comparisons based on the entropy measure and established relations between entropy comparisons and comparisons with respect to other measures in reliability. Conditions for decreasing (increasing) uncertainty in a resi...

The concept of uncertain entropy is used to provide a quantitative measurement of the uncertainty associated with uncertain variables. After introducing the definition, this paper gives some examples of entropy of uncertain variables. Furthermore this paper proposes the maximum entropy principle for uncertain variables, that is, out of all the uncertainty distributions satisfying given constrai...

The formula for the spectral moments is expanded in a series consisting of autocorrelation terms. By using the autocorrelation extrapolation inherent in the maximum entropy method (MEM) for spectral analysis, it is found that good estimates of the moments can be found from a very low order autoregressive model. The motivation for this investigation came from results obtained during spectral ana...

Traditionally, the Method of (Shannon-Kullback’s) Relative Entropy Maximization (REM) is considered with linear moment constraints. In this work, the method is studied under frequency moment constraints which are non-linear in probabilities. The constraints challenge some justifications of REM since a) axiomatic systems are developed for classical linear moment constraints, b) the feasible set ...