A Spatially AdaptiveWiener Filter for Reflectance Estimation

Wiener filtering has many applications in the area of imaging science. In image processing, for instance, it is a common way of reducing Gaussian noise. In color science it is often used to estimate reflectances from camera response data on a pixel by pixel basis. Based on a priori assumptions the Wiener filter is the optimal linear filter in the sense of the minimal mean square error to the actual data. In this paper we propose a spatially adaptive Wiener filter to estimate reflectances from images captured by a multispectral camera. The filter estimates pixel noise using local spatial neighborhood and uses this knowledge to estimate a spectral reflectance. In the hypothetical case of a noiseless system, the spatially adaptive Wiener filter equals the standard Wiener filter for reflectance estimation. We present results of various simulation experiments conducted on a multispectral image database using a 6-channel acquisition system and different noise levels. Introduction Estimation of reflectance spectra from camera responses is generally an ill-posed problem since a high dimensional signal is reconstructed from a relatively low dimensional signal. Associated with the development of multispectral camera systems many techniques were developed to tackle this problem. The basic approach to achieve a good estimation of scene reflectance spectra is to utilize as much information from the underlying capturing process as possible. Success is commonly evaluated through the spectral root mean square difference from the measured reflectance spectrum or color differences (e.g. CIEDE2000 [1]) for a set of selected illuminants. We will give a short and by far not exhaustive overview of reflectance reconstruction methods in the following text. Information used by reflectance estimation methods may include the knowledge of the acquisition illuminant, the channel sensitivities of the camera, noise properties of the system and a priori knowledge of the source reflectance. Most of the methods listed below require a priori knowledge of the spectral sensitivities of the camera system and of the acquisition illuminant. A common approach is to consider properties of natural reflectance spectra as additional a priori knowledge. These properties include positivity, boundedness and smoothness. Low effective dimensionality [2]. of natural reflectances is the reason why many methods use a low-dimensional linear model to describe spectra such as introduced in this context by Maloney and Wandell [3]. Some methods utilize a low dimensional linear model of reflectances to calculate the smoothest reflectance of all device metameric spectra (all spectra that lead to the given sensor response)[4, 5, 6]. Other methods use nearest neighbor type approaches within higher dimensional linear models [7] or adaptive principle component analysis (PCA) [8]. It was observed that a combination of multiple techniques can lead to improved reconstructions [9]. DiCarlo and Wandell extended the linear model in order to find reflectances lying on a submanifold that may describe the set of captured reflectances more accurate [10]. When camera sensitivities are not available, some approaches treat the system as a black box and use captured color-targets with known reflectances in order to construct a response-to-reflectance transformation [11, 12, 13]. The accuracy of these target-based methods is by construction highly dependent on the training target [14]. If additional information about the captured spectra is known, e.g. by a low resolution spectral sampling of the image [15] or by capturing printed images knowing the model of the printing device [16, 17], the accuracy of the spectral reconstruction can be further improved. A special linear estimation technique widely used in spectral reconstruction is the Wiener filter. Based on the assumption of a normal distribution of reflectances and system noise and the assumption that noise is statistically independent of the reflectances, it is the optimal linear filter in the sense of the minimal mean square error to the actual reflectance. The Wiener filter has the form r = KrΩ (ΩKrΩ +Kε )−1c (1) where Kr is the covariance matrix of reflectance spectra, Kε is the covariance matrix of additive noise, c is the sensor response, Ω is the device lighting matrix described in detail in eq. (2) and r is the reconstructed spectrum. Several factors can prevent the Wiener filter from performing optimally in the sense of the minimal mean square error: 1. The reflectance covariance matrix Kr can only be approximated suboptimally. A minimal knowledge approach uses a Toeplitz matrix [18]. Other approaches use a representative set of reflectances to estimate the covariance matrix [19]. Shen and Xin [20, 21] proposed a method that adaptively selects and weighs these training spectra in order to estimate the reflectance covariance matrix based on the actual sensor response. A Bayesian approach of Zhang and Brainard highly related to the Wiener filter estimates the covariance 16th Color Imaging Conference Final Program and Proceedings 279 matrix using a low dimensional space of reflectance weights [22]. 2. The Wiener filter cannot ensure the positivity and boundedness of the estimation. Both are important properties of natural reflectances. In the approach of Zhang and Brainard [22] that performs a Gaussian fit in a low dimensional space of reflectance weights all weights that correspond to reflectance functions with negative values have been excluded. This technique shall ensure the positivity of reconstructions. 3. Noise plays an important role in image acquisition systems. The accuracy of the Wiener estimation is highly dependent on the magnitude of system noise. Furthermore, the Wiener filter assumes signal-independent noise and disregards the signal-dependent shot noise. In eq. (3) the noise sources in electronic imaging devices are sketched and a commonly used noise model is introduced. An additional problem is the estimation of the noise covariance matrix. The accuracy of the Wiener reconstruction is highly dependent on the quality of the noise covariance estimation. Shimano [23] proposed a method for estimating this noise covariance matrix and achieved a good performance in terms of colorimetric and spectral RMS errors compared to multiple methods described above [24]. The first two problems above are not addressed in this paper. If desired, the algorithms proposed by other authors can be incorporated to the proposed spatially adaptive Wiener filter. Our paper focuses solely on the noise problem. The observation of the large dependency of the Wiener estimation on the magnitude of the noise variance leads to the idea of reducing noise based on the local pixel neighborhood in order to improve the reconstruction of the Wiener filter. The idea of combining spectral reflectance reconstruction with spatial noise reduction is not new. In a recent article Murakami et al. [25] proposed a spatio-spectral Wiener filter, which was called 3D Wiener (merging of 2 spatial dimension and 1 spectral dimensions in a single Wiener filter). In Murakami’s et al. article a sequence of spatial Wiener filtering followed by spectral Wiener filtering was investigated as well and called 2D+1D Wiener filter. The 2D Wiener filter is applied channel-wise on the sensor-response image. The noise variance is estimated globally from the resulting image and used in the subsequent spectral 1D Wiener filter. The approach proposed in this paper goes beyond 2D+1D Wiener filtering but does not go far as 3D Wiener filtering. In contrast to Murakami’s et al. 2D+1D approach our 2D noise reduction is performed on all channels simultaneously using a single Wiener filter. The noise covariance matrix is updated locally and propagated to the spectral Wiener filter. Both steps can be combined as a single operator, which enables a simple parallel computing, similar to Murakami’s et al. 3D Wiener filter. We will derive the spatially adaptive Wiener filter by Bayesian inference. Its noise reduction and propagation properties will be especially emphasized. Model of a Linear Acquisition System In this paper we consider linear acquisition systems. The discrete model of a n-channel capturing system is given by the following formula