Direct Importance Estimation with Gaussian Mixture Models

The ratio of two probability densities is called the importance and its estimation has gathered a great deal of attention these days since the importance can be used for various data processing purposes. In this paper, we propose a new importance estimation method using Gaussian mixture models (GMMs). Our method is an extension of the Kullback-Leibler importance estimation procedure (KLIEP), an importance estimation method using linear or kernel models. An advantage of GMMs is that covariance matrices can also be learned through an iterative estimation procedure, so the proposed method—which we call the Gaussian mixture KLIEP (GM-KLIEP)—is expected to work well when the true importance function has high correlation. Through experiments, we show the validity of the proposed approach.