We consider the problem of estimating Shannon’s entropy H from discrete data, in cases where the number of possible symbols is unknown or even countably infinite. The Pitman-Yor process, a generalization of Dirichlet process, provides a tractable prior distribution over the space of countably infinite discrete distributions, and has found major applications in Bayesian non-parametric statistics...

Some combinatorial properties of fixed boundary rhombus random tilings with octagonal symmetry are studied. A geometrical analysis of their configuration space is given as well as a description in terms of discrete dynamical systems, thus generalizing previous results on the more restricted class of codimension-one tilings. In particular this method gives access to counting formulas, which are ...

This paper derives new entropy bounds for discrete random variables via maximal coupling. It provides bounds on the difference between the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. These bounds address cases of finite or countable infinite alphabets. Particular cases of these bounds reproduce some kno...

Integrating discrete information extracted from random process solves the impulse cutting off entropy functional (EF) measure on trajectories Markov diffusion process, integrated in information path functional (IPF). Defining the EF via the process additive functional with functions drift and diffusion allows reducing this functional on trajectories to a regular integral functional. Compared to...

The algebraic flux correction (AFC) schemes presented in this work constrain a standard continuous finite element discretization of nonlinear hyperbolic problem to satisfy relevant maximum principles and entropy stability conditions. desired properties are enforced by applying limiter antidiffusive fluxes that represent the difference between high-order baseline scheme property-preserving appro...

A ranked poset is called strongly Sperner if the size of k-family cannot exceed the sum of k-largest Whitney numbers. In the sense of a function ordering, a function f is (weakly) majorized by g if the the sum of k-largest values in f cannot exceed the sum of k-largest values in g. Two definitions arise from different contexts, but they have strong similarities in their own meanings. Furthermor...

We describe five types of results concerning information and concentration of discrete random variables, and relationships between them. These results are motivated by their counterparts in the continuous case. The results we consider are information theoretic approaches to Poisson approximation, the maximum entropy property of the Poisson distribution, discrete concentration (Poincaré and loga...

For c ∈ (0, 1) let Pn(c) denote the set of n-vertex perfect graphs with density c and Cn(c) the set of n-vertex graphs without induced C5 and with density c. We show that log2 |Pn(c)|/ ( n 2 ) = log2 |Cn(c)|/ ( n 2 ) = h(c) + o(1) with h(c) = 12 if 1 4 ≤ c ≤ 3 4 and h(c) = 1 2H(|2c− 1|) otherwise, where H is the binary entropy function. Furthermore, we use this result to deduce that almost all ...

We lift important results about universally typical sets, typically sampled sets, and empirical entropy estimation in the theory of samplings of discrete ergodic information sources from the usual one-dimensional discrete-time setting to a multidimensional lattice setting. We use techniques of packings and coverings with multidimensional windows to construct sequences of multidimensional array ...

In this paper, we propose a new entropy satisfying finite volume method to solve the FokkerPlanck equation of FENE dumbbell model for polymers, subject to homogeneous fluids. Both semi-discrete and fully discrete schemes satisfy all three desired properties: i) mass conservation, ii) positivity preserving, and iii) entropy satisfying. These ensure that the computed solution is a probability den...