While entropy per unit time is a meaningful index to quantify the dynamic features of experimental time series, its estimation is often hampered in practice by the finite length of the data. We here investigate the performance of entropy estimation procedures, relying either on block entropies or Lempel-Ziv complexity, when only very short symbolic sequences are available. Heuristic analytical ...

the second law of thermodynamic (entropy principle) is of special theo-philosophical importance. it seems that this law, which is renowned as “the death of software design” and despite the order in the world, is like an incompatible universal. this article examines the concept of entropy in the three temperature, statistic, and information applications and addresses the contradiction of the sec...

In this paper we introduce the concept of entropy operator for continuous systems of finite topological entropy. It is shown that it generates the Kolmogorov entropy as a special case. If $phi$ is invertible then the entropy operator is bounded with the topological entropy of $phi$ as its norm.

Several splitting criteria for binary classification trees are shown to be written as weighted sums of two values of divergence measures. This weighted sum approach is then used to form two families of splitting criteria. One of them contains the chi-squared and entropy criterion, the other contains the mean posterior improvement criterion. Both family members are shown to have the property of ...

In the framework of multi-criteria decision making whose aggregation process is based on the Choquet integral, we present a maximum entropy like method enabling to determine, if it exists, the “least specific” capacity compatible with the initial preferences of the decision maker. The proposed approach consists in solving a strictly convex quadratic program whose objective function is equivalen...

For the task of online translation of scientific video lectures, using huge models is not possible. In order to get smaller and efficient models, we perform data selection. In this paper, we perform a qualitative and quantitative comparison of several data selection techniques, based on cross-entropy and infrequent n-gram criteria. In terms of BLEU, a combination of translation and language mod...

Multi-objective optimization evolutionary techniques provide solutions for a specific problem using optimally concepts taking into consideration all the design criteria. In the last years, several multi-objective algorithms were proposed but usually the performance is measured at the end neglecting, therefore, the solution diversity along the interactions. In order to understand the evolution o...

In this paper we show how the Shannon entropy is connected to the theory of majorization. They are both linked to the measure of disorder in a system. However, the theory of majorization usually gives stronger criteria than the entropic inequalities. We give some generalized results for majorization inequality using Csiszár f-divergence. This divergence, applied to some special convex functions...