This study critically analyses the information-theoretic, axiomatic and combinatorial philosophical bases of the entropy and cross-entropy concepts. The combinatorial basis is shown to be the most fundamental (most primitive) of these three bases, since it gives (i) a derivation for the Kullback-Leibler cross-entropy and Shannon entropy functions, as simplified forms of the multinomial distribu...

The cross-entropy method (CE) developed by R. Rubinstein is an elegant practical principle for simulating rare events. The method approximates the probability of the rare event by means of a family of probabilistic models. The method has been extended to optimization, by considering an optimal event as a rare event. CE works rather good when dealing with deterministic function optimization. Now...

Cross-entropy (CE)-based stopping criteria for turbo iterative decoding are known to outperform fixed-iteration stopping criteria at high signal-to-noise ratios (SNRs). While CE-based stopping criteria have a range of thresholds, a highvalue threshold for small frame sizes, and vice versa, should be used. It is difficult to advocate the value that can be categorized as either a small or large f...

This paper addresses the problem of resource allocation in a finite-capacity, stochastic (random) and dynamic multi-project system. The system is modeled as a queuing network that is controlled by limiting the number of concurrent projects. We propose a Cross Entropy (CE) based approach to determine near-optimal resource allocations to the entities that execute the projects. The performance of ...

We present a new generic minimum cross-entropy method, called the semiiterative MinxEnt, or simply SME, for rare-event probability estimation, counting, and approximation of the optimal solutions of a broad class of NP-hard linear integer and combinatorial optimization problems (COP’s). The main idea of our approach is to associate with each original problem an auxiliary single-constrained conv...

A derivative-free framework for optimizing a non-smooth range-based contrast function in order to estimate independent components is presented. The proposed algorithm employs the von-Mises Fisher (vMF) distribution to draw random samples in the cross-entropy (CE) method, thereby intrinsically maintaining the unit-norm constraint that removes the scaling indeterminacy in independent component an...

In this article, challenges regarding the provision of channel state information (CSI) in non-contiguous orthogonal frequency division multiplexing (NC-OFDM) cognitive radio (CR) systems are addressed. We propose a novel scheme that utilizes cross entropy (CE) optimization together with an analytical pilot power distribution technique to design pilot symbols that minimizes the channel estimate ...

The cross entropy (CE) method is a model based search method to solve optimization problems where the objective function has minimal structure. The Monte-Carlo version of the CE method employs the naive sample averaging technique which is inefficient, both computationally and space wise. We provide a novel stochastic approximation version of the CE method, where the sample averaging is replaced...

The problem in estimating a social accounting matrix (SAM) for a recent year is to find an efficient and cost-effective way to incorporate and reconcile information from a variety of sources, including data from prior years. Based on information theory, the paper presents a flexible “cross entropy” (CE) approach to estimating a consistent SAM starting from inconsistent data estimated with error...

This paper introduces a novel solver, namely cross entropy (CE), into the MRF theory for medical image segmentation. The solver, which is based on the theory of rare event simulation, is general and stochastic. Unlike some popular optimization methods such as belief propagation and graph cuts, CE makes no assumption on the form of objective functions and thus can be applied to any type of MRF m...