The Generalized Cross Entropy Method, with Applications to Probability Density Estimation
نویسندگان
چکیده
Nonparametric density estimation aims to determine the sparsest model that explains a given set of empirical data and which uses as few assumptions as possible. Many of the currently existing methods do not provide a sparse solution to the problem and rely on asymptotic approximations. In this paper we describe a framework for density estimation which uses information-theoretic measures of model complexity with the aim of constructing a sparse density estimator that does not rely on large sample approximations. The effectiveness of the approach is demonstrated through an application to some well-known density estimation test cases. AMS2000 subject classifications:Primary94A17, 60K35; secondary68Q32, 93E14.
منابع مشابه
Generalized Cross-entropy Methods with Applications to Rare-event Simulation and Optimization
The cross-entropy and minimum cross-entropy methods are well-known Monte Carlo simulation techniques for rare-event probability estimation and optimization. In this paper, we investigate how these methods can be extended to provide a general non-parametric cross-entropy framework based on 1-divergence distance measures. We show how the 2 2 distance, in particular, yields a viable alternative to...
متن کاملGeneralized Cross-Entropy Methods
The cross-entropy and minimum cross-entropy methods are well-known Monte Carlo simulation techniques for rare-event probability estimation and optimization. In this paper we investigate how these methods can be extended to provide a general non-parametric cross-entropy framework based on φ-divergence distance measures. We show how the χ distance in particular yields a viable alternative to Kull...
متن کاملModeling of the Maximum Entropy Problem as an Optimal Control Problem and its Application to Pdf Estimation of Electricity Price
In this paper, the continuous optimal control theory is used to model and solve the maximum entropy problem for a continuous random variable. The maximum entropy principle provides a method to obtain least-biased probability density function (Pdf) estimation. In this paper, to find a closed form solution for the maximum entropy problem with any number of moment constraints, the entropy is consi...
متن کاملDischarge Estimation by using Tsallis Entropy Concept
Flow-rate measurement in rivers under different conditions is required for river management purposes including water resources planning, pollution prevention, and flood control. This study proposed a new discharge estimation method by using a mean velocity derived from a 2D velocity distribution formula based on Tsallis entropy concept. This procedure is done based on several factors which refl...
متن کاملA non-asymptotic bandwidth selection method for kernel density estimation of discrete data
In this paper we explore a method for modeling of categorical data derived from the principles of the Generalized Cross Entropy method. The method builds on standard kernel density estimation techniques by providing a novel non-asymptotic data-driven bandwidth selection rule. In addition to this, the Entropic approach provides model sparsity not present in the standard kernel approach. Numerica...
متن کامل