Two mirror symmetric versions of the maximum entropy (MaxEnt) methodology are introduced and compared: (1) A posteriori MaxEnt Independent Component Analysis (ICA) H(V) was proposed by Bell, Sejnowski, Amari, Oja (BSAO) (early by Jutten & Herault, Comon and Cardoso (JHCC) in France). It is ambitious to factorize the unknown joint-probability density function (j-pdf) using the post processing al...

This paper proposes a new method for blind inversion of a monotonic nonlinear map applied to a sum of random variables. Such kinds of mixtures of random variables are found in source separation and Wiener system inversion problems, for example. The importance of our proposed method is based on the fact that it permits to decouple the estimation of the nonlinear part (nonlinear compensation) fro...

Degree distributions have been widely used to characterize biological networks including food webs, and play a vital role in recent models of food web structure. While food webs degree distributions have been suggested to follow various functional forms, to date there has been no mechanistic or statistical explanation for these forms. Here I introduce models for the degree distributions of food...

Maximum entropy (MaxEnt) models have been used in many spoken language tasks. The training of a MaxEnt model often involves an iterative procedure that starts from an initial parameterization and gradually updates it towards the optimum. Due to the convexity of its objective function (hence a global optimum on a training set), little attention has been paid to model initialization in MaxEnt tra...

We present a new class of density estimation models, Structural Maxent models, with feature functions selected from a union of possibly very complex sub-families and yet benefiting from strong learning guarantees. The design of our models is based on a new principle supported by uniform convergence bounds and taking into consideration the complexity of the different sub-families composing the f...

We give an interpretation of the Maxi­ mum Entropy (MaxEnt) Principle in game­ theoretic terms. Based on this interpretation, we make a formal distinction between differ­ ent ways of applying Maximum Entropy dis­ tributions. MaxEnt has frequently been crit­ icized on the grounds that it leads to highly representation dependent results. Our dis­ tinction allows us to avoid this problem in many c...

This paper presents a hybrid approach for named entity (NE) tagging which combines Maximum Entropy Model (MaxEnt), Hidden Markov Model (HMM) and handcrafted grammatical rules. Each has innate strengths and weaknesses; the combination results in a very high precision tagger. MaxEnt includes external gazetteers in the system. Sub-category generation is also discussed.