Hyperparameter estimation in image restoration

The hyperparameter in image restoration by the Bayes formula is an important quantity. This communication shows a physical method for the estimation of the hyperparameter without approximation. For artificially generated images by prior probability, the hyperparameter is computed accurately. For practical images, accuracy of the estimated hyperparameter depends on the magnetization and energy of the images. We discuss the validity of prior probability for an original image. PACS numbers: 02.50.−r, 05.50.+q, 07.05.Pj, 95.75.Mn Mathematical methods in statistical physics have been applied to information processing problems [1]. A probabilistic model has been used to construct the problems. An analogy between a probabilistic model and a formula of statistical physics supports the validity for applicability. One of the main topics which deals with statistical-physics approaches to information processing problems is image restoration [2]. An image on a computer is represented by a sequence of bits. When a digital image is transferred through a channel, it is corrupted by noise. The purpose of image restoration is to restore an original image from a degraded image. The problem is to infer an original image from a degraded image. The Bayes formula plays an important role in the image restoration by a probabilistic method [3]. The Bayes formula is expressed by (posterior prob.) ∝ (conditional prob.) ∗ (prior prob.). The conditional probability and the prior probability contain parameters (hyperparameters). In order to obtain a properly restored image by the posterior probability of the Bayes formula, it is necessary to use appropriate values of the hyperparameters [4]. However, one usually has only a degraded image and no knowledge of a degradation process characterized by the conditional probability and an original image characterized by the prior probability. For the sake of simplicity, we assume that the conditional probability is given by a memoryless symmetric channel. We have to estimate the hyperparameters from a degraded image [5–9]. So far the hyperparameters are determined by maximizing a marginal likelihood function (MML) [10]. However, a computational task for the summation in marginalization is exponentially huge. One has to resort to simulation or approximate methods (mean-field or Bethe approximation) to 1751-8113/08/332004+07$30.00 © 2008 IOP Publishing Ltd Printed in the UK 1 J. Phys. A: Math. Theor. 41 (2008) 332004 Fast Track Communication implement the method. In the present communication, we demonstrate a physical method for obtaining the hyperparameters without any approximations. Morita and Tanaka attempted to estimate the hyperparameters on an additional assumption about images [11]. By comparison with their method, our method is simple and natural. We consider a binary (black and white) image. A black pixel represents 0 as bit expression or down spin in the Ising model. A white pixel represents 1 as bit expression or up spin in the Ising model. When an original image is corrupted by noise, one receives a degraded image with the state of a pixel inverted from an original value with probability p. In a binary symmetric channel a change from the state of each pixel to another state occurs with the same probability p independently of the other pixels. The probability p is one of the hyperparameters. We examine a change of the number of black or white pixels between an original image and a degraded image: { N ′ 1 = (1 − p)N1 + pN0 N ′ 0 = pN1 + (1 − p)N0, (1) where N1 and N0 are the number of white and black pixels in an original image, respectively. The prime means the number of pixels in a degraded image. Solving the eigenvalue problem, we obtain the following relations: N ′ 1 + N ′ 0 = N1 + N0, (2) N ′ 1 −N ′ 0 = (1 − 2p)(N1 −N0). (3) Equation (2) shows conservation of the total number of pixels between an original image and a degraded image. From the point of view of Ising spin, Equation (3) means a change of magnetization, which is defined by the difference between the number of sites with up spin and that with down spin. Equation (3) can be expressed in the following way, M ′ = (1 − 2p)M, (4) where M andM ′ are the magnetization of an original image and a degraded image, respectively. We consider a change of the number of neighboring-pixel pairs between an original image and a degraded image: ⎧⎪⎨ ⎪⎩ N ′ 11 = (1 − p)N11 + p(1 − p)N10 + pN00 N ′ 10 = 2p(1 − p)N11 + [(1 − p)2 + p]N10 + 2p(1 − p)N00 N ′ 00 = pN11 + p(1 − p)N10 + (1 − p)N00, (5) where N11 is the number of neighboring-pixel pairs with 1 as a bit on both ends. Similarly N10 and N00 are defined. The prime means those in a degraded image. Solving the eigenvalue problem, we obtain the following relations, N ′ 11 + N ′ 10 + N ′ 00 = N11 + N10 + N00, (6) N ′ 11 −N ′ 00 = (1 − 2p)(N11 −N00), (7) N ′ 11 −N ′ 10 + N ′ 00 = (1 − 2p)(N11 −N10 + N00). (8) The first expression (6) shows conservation of the total number of neighboring pairs between an original image and a degraded image. The formula (N11−N10 +N00) in the third expression (8) is equal to the minus energy of the Ising model, −H = ∑〈i,j〉 σiσj . Equation (8) can be expressed in the following way, E′ = (1 − 2p)E, (9)