Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy rates—measures of new disorder per new observed value—are equal for ergodic finite-alphabet information sources (discrete-time stationary stochastic processes). With this...

We propose a thermodynamic interpretation of transfer entropy near equilibrium, using a specialised Boltzmann’s principle. The approach relates conditional probabilities to the probabilities of the corresponding state transitions. This in turn characterises transfer entropy as a difference of two entropy rates: the rate for a resultant transition and another rate for a possibly irreversible tra...

This paper aims to perform the second law analysis of thermodynamics on the laminar film condensation of pure saturated vapor flowing in the direction of gravity on an ellipsoid with variable wall temperature. The analysis provides us understanding how the geometric parameterellipticity and non-isothermal wall temperature variation amplitude “A.” affect entropy generation during film-wise conde...

Long memory or long range dependency is an important phenomenon that may arise in the analysis of time series or spatial data. Most of the definitions of long memory of a stationary process X = {X1, X2, · · · , } are based on the second-order properties of the process. The excess entropy of a stationary process is the summation of redundancies which relates to the rate of convergence of the con...

Statistical Energy Analysis is a statistical method in vibroacoustics entirely based on the application of energy balance that is the first principle of thermodynamics. In this study, the definition of vibrational entropy is introduced for sub-systems containing energy and modes. The rate of entropy production at interfaces between sub-systems is also derived. Finally, in steady-state condition...

We establish upper and lower bounds for the metric entropy and bracketing entropy of the class of d-dimensional bounded monotonic functions under L norms. It is interesting to see that both the metric entropy and bracketing entropy have different behaviors for p < d/(d − 1) and p > d/(d − 1). We apply the new bounds for bracketing entropy to establish a global rate of convergence of the MLE of ...

Let KpX1, . . . , Xnq and HpXn|Xn ́1, . . . , X1q denote the Kolmogorov complexity and Shannon’s entropy rate of a stationary and ergodic process tXiui“ ́8. It has been proved that KpX1, . . . , Xnq n ́HpXn|Xn ́1, . . . , X1q Ñ 0, almost surely. This paper studies the convergence rate of this asymptotic result. In particular, we show that if the process satisfies certain mixing conditions, then the...

We generalize Gaspard's method for computing the ε-entropy production rate in Hamiltonian systems to dissipative systems with attractors considered earlier by Tél, Vollmer, and Breymann. This approach leads to a natural definition of a coarse grained Gibbs entropy which is extensive, and which can be expressed in terms of the SRB measures and volumes of the coarse graining sets which cover the ...

In this work, we extend a variable-length source coding theorem for discrete memoryless sources to ergodic time-invariant Markov sources of arbitrary order. To accomplish this extension, we establish a formula for the R enyi entropy rate lim n!1 H (n)=n. The main tool used to obtain the R enyi entropy rate result is Perron-Frobenius theory. We also examine the expression of the R enyi entropy r...

We present here two methods of entropy coding for the lattice codevectors. We compare our entropy coding methods with one method previously presented in the literature from the point of view of rate-distortion as well as of the computation complexity and memory requirements. The results are presented for artificial Laplacian and Gaussian data, as well as for LSF parameters of speech signals. In...