The Shannon lower bound is one of the few lower bounds on the rate-distortion function that holds for a large class of sources. In this paper, it is demonstrated that its gap to the rate-distortion function vanishes as the allowed distortion tends to zero for all sources that have a finite differential entropy and whose integer parts have a finite entropy. Conversely, it is demonstrated that if...

Csiszár’s forward -cutoff rate (given a fixed 0) for a discrete source is defined as the smallest number such that for every , there exists a sequence of fixed-length codes of rate with probability of error asymptotically vanishing as . For a discrete memoryless source (DMS), the forward -cutoff rate is shown by Csiszár [6] to be equal to the source Rényi entropy. An analogous concept of revers...

{The performance of optimum vector quantizers subject to a conditional entropy constraint is studied in this paper. This new class of vector quantizers was originally suggested by Chou and Lookabaugh. A locally optimal design of this kind of vector quantizer can be accomplished through a generalization of the well known entropy-constrained vector quantizer (ECVQ) algorithm. This generalization ...

This paper deals with the discrimination between suffering foetuses and normal foetuses by means of a multi-scale similarity entropy. Sample entropy and similarity entropy are evaluated in multi-scale analysis on foetal heart rate signals. Without multi-scale analysis, our results show that only the similarity entropy differentiate suffering foetuses to normal foetuses. Furthermore with the mul...

Distributed Denial of Service (DDoS) attacks represent a major threat to uninterrupted and efficient Internet service. In this paper, we empirically evaluate several major information metrics, namely, Hartley entropy, Shannon entropy, Renyi’s entropy, generalized entropy, Kullback–Leibler divergence and generalized information distance measure in their ability to detect both low-rate and high-r...

An approach to produce a model for the data generating distribution is the well-known maximum entropy method. In this approach, the partial knowledge about the data generating distribution is formulated in terms of a set of information constraints, usually moment constraints, and the inference is based on the model that maximizes Shannon’s entropy under these constraints. In this paper we inves...

The objective of this study is to investigate the thermal efficiency and power production of typical models of endoreversible heat engines at the regime of minimum entropy generation rate. The study considers the Curzon-Ahlborn engine, the Novikov's engine, and the Carnot vapor cycle. The operational regimes at maximum thermal efficiency, maximum power output and minimum entropy production rate...

Entropy functionals are computed for non-stationary distributions of particles of Lorentz gas and hard disks. The distributions consisting of beams of particles are found to have the largest amount of entropy and entropy increase. The computations show exponentially monotonic increase during initial time of rapid approach to equilibrium. The rate of entropy increase is bounded by sums of positi...

Cells can be considered as systems that utilize changes in thermodynamic entropy as information. Therefore, they serve as useful models for investigating the relationships between entropy production and information transmission, i.e., signal transduction. Based on the hypothesis that cells apply a chemical reaction cascade for the most efficient transduction of information, we adopted a coding ...

The physical mechanisms of entropy generation in a condenser with high fractions of noncondensable gases are examined using scaling and boundary layer techniques, with the aim of defining a criterion for minimum entropy generation rate that is useful in engineering analyses. This process is particularly relevant in humidification-dehumidification desalination systems, where minimizing entropy g...