Entropy of Weight Distributions of Small-Bias Spaces and Pseudobinomiality

A classical bound in information theory asserts that small L1-distance between probability distributions implies small difference in Shannon entropy, but the converse need not be true. We show that if a probability distribution on {0, 1} has small-bias, then the converse holds for its weight distribution in the proximity of the binomial distribution. Namely, we argue that if a probability distribution μ on {0, 1} is δ-biased, then ‖μ− binn ‖1 ≤ (2 ln 2)(nδ +H(binn)− H(μ)), where μ is the weight distribution of μ and binn is the binomial distribution on {0, . . . , n}. The key result behind this bound is a lemma which asserts the non-positivity of all the Fourier coefficients of the log-binomial function L : {0, 1} → R given by L(x) = lg binn(|x|). The original question which motivated the work reported in this paper is the problem of explicitly constructing a small subset of {0, 1} which is -pseudobinomial in the sense that the weight distribution of each of its restrictions and translations is -close to the binomial distribution. We study the notion of pseudobinomiality and we conclude that, for spaces with n−Θ(1)-small bias, the pseudobinomiality error in the L1-sense is equivalent to that in the entropy-difference-sense, in the n−Θ(1)-error regime. We also study the notion of average case pseudobinomiality, and we show that for spaces with n−Θ(1)-small bias, the average entropy of the weight distribution of a random translation of the space is n−Θ(1)-close to the entropy of the binomial distribution. We discuss resulting questions on the pseudobinomiality of sums of independent small-bias spaces. Using the above results, we show that the following conjectures are equivalent: (1) For all independent δ-biased random vectors X,Y ∈ {0, 1}, the F2-sum X + Y is O((nδ))pseudobinomial; (2) For all independent δ-biased random vectors X,Y ∈ {0, 1}, the entropy of the weight of the sum H(|X + Y |) ≥ min{H(|X|), H(|Y |)} −O((nδ)). ∗To appear in Chicago Journal of Theoretical Computer Science, 2015 †A conference version of this paper appeared in COCOON 2015, LNCS 9198 proceedings, pages 495-506. ‡Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon. E-mail: [email protected].