In this paper we present a new training algorithm for the Long Short-Term Memory (LSTM) recurrent neural network. This algorithm uses entropy instead of the usual mean squared error as the cost function for the weight update. More precisely we use the Error Entropy Minimization approach, were the entropy of the error is minimized after each symbol is present to the network. Our experiments show...

Consider a de ned density on a set of very large dimension. It is quite di cult to nd an estimate of this density from a data set. However, it is possible through a projection pursuit methodology to solve this problem. In his seminal article, Huber (see "Projection pursuit", Annals of Statistics, 1985) demonstrates the interest of his method in a very simple given case. He considers the factori...

The paper provides a numerical investigation of the entropy generation analysis due to natural convection in an inclined square porous cavity. The coupled equations of mass, momentum, energy and species conservation are solved using the Control Volume Finite-Element Method. Effect of medium permeability and inclination angle on entropy generation is analysed. It was found that according to the ...

We consider the semi-supervised learning problem, where a decision rule is to be learned from labeled and unlabeled data. In this framework, we motivate minimum entropy regularization, which enables to incorporate unlabeled data in the standard supervised learning. Our approach includes other approaches to the semi-supervised problem as particular or limiting cases. A series of experiments illu...

This paper considers the minimization of a convex integral functional over the positive cone of an Lp space, subject to a finite number of linear equality constraints. Such problems arise in spectral estimation, where the objective function is often entropy-like, and in constrained approximation. The Lagrangian dual problem is finite-dimensional and unconstrained. Under a quasi-interior constra...

Given iid samples drawn from a distribution with known parametric form, we propose the minimization of expected Bregman divergence to form Bayesian estimates of differential entropy and relative entropy, and derive such estimators for the uniform, Gaussian, Wishart, and inverse Wishart distributions. Additionally, formulas are given for a log gamma Bregman divergence and the differential entrop...

Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to understand individually sparsity or entropy from specific research interests. This paper proposes a mathematical formalism, a joint axiomatic characterization, ...

This article describes the generation of a detailed two-layered synthetic population households and individuals for French municipalities. Using census data, four reconstruction methods associated with two probabilistic integerization are applied. The paper offers an in-depth description each method through common framework. A comparison these is then carried out on basis various criteria. Resu...

We discuss a special class of generalized divergence measures by the use of generator functions. Any divergence measure in the class is separated into the difference between cross and diagonal entropy. The diagonal entropy measure in the class associates with a model of maximum entropy distributions; the divergence measure leads to statistical estimation via minimization, for arbitrarily giving...