In the framework of information dynamics, the temporal evolution of coupled systems can be studied by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it. While information storage and transfer are computed through the known self-entropy (SE) and transfer entropy (TE), an a...

This paper presents a novel hybrid algorithm for combinatorial optimization problems based on mixing the cross-entropy (CE) method and a Hopfield neural network. The algorithm uses the CE method as a global search procedure, whereas the Hopfield network is used to solve the constraints associated to the problems. We have shown the validity of our approach in several instance of the generalized ...

The three main theoretical bases of the concepts of entropy and cross-entropy informationtheoretic, axiomatic and combinatorial are critically examined. It is shown that the combinatorial basis, proposed by Boltzmann and Planck, is the most fundamental (most primitive) basis of these concepts, since it provides (i) a derivation of the Kullback-Leibler cross-entropy and Shannon entropy functions...

This paper proposes a method for the measurement and quantification of the impact of accents on speech models. An accent metric is introduced based on the cross entropy (CE) of the probability models of speech from different accents. The CE metric has potentials for use in analysis, identification, quantification and ranking of the salient features of accents. The accent metric is used for phon...

We present two randomized entropy-based algorithms for approximating quite general #P-complete counting problems, like the number of Hamiltonian cycles in a graph, the permanent, the number of self-avoiding walks and the satisfiability problem. In our algorithms we first cast the underlying counting problem into an associate rare-event probability estimation, and then apply dynamic importance s...

The variance minimization (VM) and cross-entropy (CE) methods are two versatile adaptive importance sampling procedures that have been successfully applied to a wide variety of difficult rare-event estimation problems. We compare these two methods via various examples where the optimal VM and CE importance densities can be obtained analytically. We find that in the cases studied both VM and CE ...

Based on an intuitionistic knowledge content nature of information gain, the concept of combination entropy CE(A) in incomplete information systems is first introduced, and some of its important properties are given. Then, the conditional combination entropy CE(Q | P ) and the mutual information CE(P ;Q) are defined. Unlike all existing measures for the uncertainty in incomplete information sys...

Partial Transmit Sequence technique is one of the most popular technique for PAPR reduction in OFDM. But in this scheme the complexity for searching the phase factor is increased with increase in number of sub blocks. This paper describes the different types of optimization methods used in PTS scheme, for optimization of phase factor & reducing the searching complexity .Also we can analyze thei...

The cross-entropy (CE) method is an adaptive importance sampling procedure that has been successfully applied to a diverse range of complicated simulation problems. However, recent research has shown that in some high-dimensional settings, the likelihood ratio degeneracy problem becomes severe and the importance sampling estimator obtained from the CE algorithm becomes unreliable. We consider a...

In this paper, cross entropy method is used for solving dual Lagrange support vector machine (SVM). Cross entropy (CE) method is a new practical approach which is widely used in some applications such as combinatorial optimization, learning algorithm and simulation. Our approach refers to Kernel Adatron which is solving dual Lagrange SVM using gradient ascent method. Hereby, the cross entropy m...