This chapter describes how difficult statistical estimation problems can often be solved efficiently by means of the cross-entropy (CE) method. The CE method can be viewed as an adaptive importance sampling procedure that uses the cross-entropy or Kullback–Leibler divergence as a measure of closeness between two sampling distributions. The CE method is particularly useful for the estimation of ...

We present a new method, called the minimum cross-entropy (MCE) method for approximating the optimal solution of NP-hard combinatorial optimization problems and rare-event probability estimation, which can be viewed as an alternative to the standard cross entropy (CE) method. The MCE method presents a generic adaptive stochastic version of Kullback’s classic MinxEnt method. We discuss its simil...

We apply the cross-entropy (CE) method to problems in clustering and vector quantization. The CE algorithm involves the following iterative steps: (a) the generation of clusters according to a certain parametric probability distribution, (b) updating the parameters of this distribution according to the Kullback-Leibler cross-entropy. Through various numerical experiments we demonstrate the high...

Based on information entropy theory, a novel feature extraction algorithm based on improved polynomial entropy (IPE) is set up. Firstly, the concepts and their properties of information entropy and cross entropy (CE) are analysed and studied. On this foundation, we prove that symmetrical cross entropy (SCE) proposed here based on CE satisfies three axioms of the distance, i.e. nonnegativity, sy...

Two stochastic optimization methods: Cross Entropy (CE) and Parametric Minimum Cross Entropy (PME) were tested against Large Max-Cut problems. The problems were taken from the "The DIMACS Library of Mixed Semidefinite-Quadratic-Linear Programs" challenge web site. The two methods achieved close and even better results than the Best Known Solution. In addition to results presentation, several im...

The cross-entropy (CE) method is a new generic approach to combinatorial and multi-extremal optimization and rare event simulation. The purpose of this tutorial is to give a gentle introduction to the CE method. We present the CE methodology, the basic algorithm and its modifications, and discuss applications in combinatorial optimization and machine learning.

In most applications of optical computed tomography (OpCT), limited-view problems are often encountered, which can be solved to a certain extent with typical OpCT reconstructive algorithms. The concept of entropy first emerged in information theory has been introduced into OpCT algorithms, such as maximum entropy (ME) algorithms and cross entropy (CE) algorithms, which have demonstrated their s...

This paper addresses the use of a stochastic optimization method called the Cross Entropy (CE) Method in the improvement of a recently proposed H2MLVQ (Harmonic to minimum LVQ ) algorithm , this algorithm was proposed as an initialization insensitive variant of the well known Learning Vector Quantization (LVQ) algorithm. This paper has two aims, the first aim is the use of the Cross Entropy (CE...

In this paper, we propose a Cross Entropy (CE) [1] based multiagent approach for solving static/dynamic traffic assignment problems (TAP). This algorithm utilizes a family of probability distributions in order to guide travelers (agents) to network equilibrium. The route choice probability distribution depends on the average network performance experienced by agents on previous days. Based on t...

This paper presents two simple and effective criteria for stopping the iteration process in turbo decoding with a negligible degradation of the error performance. Both criteria are devised based on the cross-entropy (CE) concept. They are as efficient as the CE criterion, but require much less and simpler computations.