Image/Video Scaling Algorithm based on Multirate Signal Processing

This paper presents an approach for image and video scaling using multirate signal processing. The main objective is to scale images with arbitrary rational scaling ratio without visible aliasing or distortion artifact. The approach can be applied to grey scale images, color images and video signals in both spatial domain and time domain. Cosine modulation is used to minimize the required on-chip memory since only a prototype filter is stored with some cosine modulation factors. The filters are shown to have comparable regularity for each scaling factor. An efficient structure is proposed for limited bit length filter coefficients which has no imaging artifact after the filter coefficients are quantization. Simulations on still image and video scaling are presented. 1. A REVIEW OF IMAGE/VIDEO SCALING Digital image/video processing is an active research area for multimedia applications due to the efficiency of digital representation of signals. Image/video scaling is desirable in system with different display resolutions. Figure 1 shows a structure for scaling an input signal x(n) with a scaling ratio L/M . Without loss of generality, we assume that gcd(L, M) = 1 where gcd is the greatest common divisor, i.e. AL+BM = 1 for some integers A and B. The input L M L,M H (z) x(n) u(n) v(n) y(n) Figure 1: A block diagram for L/M image scaler signal x(n) is upsampled by L, filtered by an interpolation filter HL,M (z), and downsampled by M . Hence the size of the output y(n) is L/M of the size of the input x(n). The relation among x(n), u(n), v(n) and y(n) can be summarized as follow [1]: u(n) = x(n/L) if L|n 0 otherwise ⇐⇒ U(z) = X(z) (1) v(n) = h(n) ∗ u(n) ⇐⇒ V (z) = H(z)U(z) (2) y(n) = v(Mn) ⇐⇒ Y (z) = 1 M M−1 X k=0 V (ez ) (3) Consequently y(n) = X k∈Z h(k)u(Mn− k) = X m∈Z h(Mn− Lm)x(m). (4) Note that from the right-hand side of Eq (4), effectively, the length of the filter is reduced by a factor of 1 L , and thus the number of multiplier needed for the computation is reduced. Such a computation can be implemented using the polyphase structure [1]. Although the block diagram presented in Figure 1 is for one dimensional processing, the same method can be applied in on image where the horizontal and vertical directions are scaled independently. Furthermore, color image scaling can also be done by using similar algorithm on the color components either in the original domain (R-G-B) or in the transformed domain (Y-I-Q). The advantage of processing in the Y-I-Q domain is that one could reduce the computation requirement by processing the decimated I and Q components. Not only can the spatial directions be processed using the system in Figure 1, but also it can be scaled along the time axis. This is an application for frame rate conversion for video signals. There are two major artifacts in multirate signal processing: aliasing and imaging artifacts from downsampling and upsampling respectively [2]. These artifacts degrade the image quality and must be suppressed in the algorithm. Notice that the frequency characteristics of the interpolation filter HL,M (z) depends on the values of L and M . From Eq (1), upsampling-by-L introduces imaging components at frequency 2πk L , 1 ≤ k ≤ L − 1 and thus the upsampled signal u(n) has to be filtered by a lowpass filter with cutoff frequency of π L . On the other hand, from Eq (3), downsampling-by-M also introduces aliasing artifact by expanding the signal spectrum of v(n) by a factor of M . Therefore an anti-aliasing lowpass filter with cutoff frequency of π M is required. Figure 2(a) and (b) show the frequency spectra of the signals for the case when L > M and L < M respectively. When L > M (Figure 2(a)), the cutoff frequency of HL,M (z) has to be π L to avoid the imaging artifact and thus the aliasing is automatically suppressed. If L < M (Figure 2(b)), the cutoff frequency of HL,M (z) has to be π M ; otherwise the imaging artifact will be created. Therefore the cutoff frequency of HL,M (z) must be π max(L,M) . In practice, implementing ideal filters is impossible, and therefore the transition band of each filter can not be zero. The nonzero stopband attenuation of the filter will pick up some energy around