Regression with Input - dependent Noise : A Gaussian Process

Gaussian processes provide natural non-parametric prior distributions over regression functions. In this paper we consider regression problems where there is noise on the output, and the variance of the noise depends on the inputs. If we assume that the noise is a smooth function of the inputs, then it is natural to model the noise variance using a second Gaussian process, in addition to the Gaus-sian process governing the noise-free output value. We show that the posterior distribution of the noise rate can be sampled using Gibbs sampling. Our results on a synthetic data set give a posterior variance that well-approximates the true variance. We also show that the predictive likelihood of a test data set approximates the true likelihood better under this model than under a uniform noise model. A very natural approach to regression problems is to place a prior on the kinds of function that we expect, and then after observing the data to obtain a posterior. The prior can be obtained by placing prior distributions on the weights in a neural network, although we would argue that it is perhaps more natural to place priors directly over functions. One tractable way of doing this is to create a Gaussian process prior. This has the advantage that predictions can be made from the posterior using only matrix multiplication for xed hyperparameters and a global noise level. In