Approximate Bayesian Recursive Estimation On Approximation Errors

Adaptive systems rely on recursive estimation of a firmly bounded complexity. As a rule, they have to use an approximation of the posterior probability density function (pdf), which comprises unreduced information about the estimated parameter. In recursive setting, the latest approximate pdf is updated using the learnt system model and the newest data and then approximated. The fact that approximation errors may accumulate over time course is mostly neglected in the estimator design and, at most, checked ex post. The paper inspects this problem and concludes that a sort of forgetting (flattening) is an indispensable part of approximate recursive estimation algorithms. The conclusion results from Bayesian paradigm complemented by the minimum cross-entropy (also known as Kullback-Leibler divergence, KLD) principle. Claims of the paper are illustrated on approximate recursive estimation of the mode and scaling factor of Cauchy pdf.