We establish upper and lower bounds for the metric entropy and bracketing entropy of the class of d-dimensional bounded monotonic functions under L norms. It is interesting to see that both the metric entropy and bracketing entropy have different behaviors for p < d/(d − 1) and p > d/(d − 1). We apply the new bounds for bracketing entropy to establish a global rate of convergence of the MLE of ...

Information entropy is one of crucial concepts in modern information theory and related fields. Over the years a large number of methods for reliable estimation of information entropy have been invented. In this paper we introduce a method for the estimation entropy of binary streams based on binary decision diagrams. Binary decision diagrams are an efficient method of representation of discret...

We obtain an estimate of free entropy of generators in a type II1-factor M which has a subfactor N of finite index with a subalgebra P = P1∨P2 ⊂ N where P1 = R1∩P , P2 = R2∩P are diffuse, R1,R2 ⊂ P are mutually commuting hyperfinite subfactors, and an abelian subalgebra A ⊂ N such that the correspondence PL 2(N , τ)A is M-weakly contained in a subcorrespondence PHA of PL 2(M, τ)A, generated by ...

In this paper, the continuous optimal control theory is used to model and solve the maximum entropy problem for a continuous random variable. The maximum entropy principle provides a method to obtain least-biased probability density function (Pdf) estimation. In this paper, to find a closed form solution for the maximum entropy problem with any number of moment constraints, the entropy is consi...

Abstract. Estimating the entropy of a sample set is required in solving numerous learning scenarios involving information theoretic optimization criteria. A number of entropy estimators are available in the literature; however, these require a batch of samples to operate on in order to yield an estimate. In this paper, we derive a recursive formula to estimate Renyi’s quadratic entropy on-line,...