This memo contains proofs that the Shannon entropy is the limiting case of both the Renyi entropy and the Tsallis entropy, or information. These results are also confirmed experimentally. We conclude with some general observations on the utility of entropy measures. A brief summary of the origins of the concept of physical entropy are provided in an appendix.

In a recent paper [1], Leonenko, Pronzato and Savani consider the estimation of the Rényi and Tsallis entropies, respectively, H ∗ q = 1 1−q log ∫ Rm f (x) dx and Hq = 1 q−1(1 − ∫ Rm f (x) dx), q = 1, of an unknown probability measure with density f on R with respect to the Lebesgue measure though kth nearestneighbor distances in a sample X1, . . . ,XN i.i.d. with the density f . The results in...