Ef ciency of Entropy Testing

—Recently it was shown that Shannon entropy is more Bahadur ef cient than any Rényi entropy of order > 1: In this paper we shall show that relative Bahadur ef ciency between any two Rényi entropies of orders 2 ]0; 1] is 1 when the relative Bahadur ef ciency is de ned according to [1]. Despite the fact that the relative Bahadur ef ciency is 1 it is shown that in a certain sense Shannon entropy is more ef cient than Rényi entropy for 2 ]0; 1[ : This indicates that the de nition of relative ef ciency given in [1] does not fully capture the notion of ef ciency. I. POWER DIVERGENCE STATISTICS Let M(k) denote the set of all discrete probability distributions of the form P = (p1; :::; pk) and M(kjn) the subset of possible types. One of the fundamental problems of mathematical statistics can be described as follows. Consider n balls distributed into bins 1; :::; k independently according to an unknown probability distribution Pn 2 M (k), which may depend on the number of balls n. This results in frequency counts Xn1; : : : ; Xnk the vector of which Xn = (Xn1; : : : ; Xnk) 2 f0; 1; : : :g is multinomially distributed with parameters k, n and Pn. The problem is to decide on the basis of observations Xn whether the unknown law Pn is equal to a given Q = (q1; :::; qk) 2M (k) or not. The observations Xn are represented by the empirical distribution P̂n = p̂n1 4 = Xn1=n; :::; p̂nk 4 = Xnk=n 2M(kjn) (1) and a procedure T on accepting or rejecting a hypothesis based on P̂n is called a test. The test uses a statistic Tn(P̂n; Q); which characterizes the goodness-oft between the distributions P̂n and Q. The test T rejects the hypothesis Pn = Q if T = Tn(P̂n; Q) exceeds a certain level rn 2 R. The goodness-oft statistic is usually one of the power divergence statistics T = T ;n = 2nD (P̂n; Q); 2 R: (2) where D (P;Q) denotes the power divergence of order of the distributions P;Q 2 M (k) de ned by