Estimation of Large Scalings in Images Based on Multilayer Pseudopolar Fractional Fourier Transform

Abstract

Accurate estimation of the Fourier transform in log-polar coordinates is a major challenge for phase-correlation based motion estimation. To acquire better image registration accuracy, a method is proposed to estimate the log-polar coordinates coefficients using multilayer pseudopolar fractional Fourier transform (MPFFT). The MPFFT approach encompasses pseudopolar and multilayer techniques and provides a grid which is geometrically similar to the log-polar grid. At low coordinates coefficients the multilayer pseudopolar grid is dense, and at high coordinates coefficients the grid is sparse. As a result, large scalings in images can be estimated, and better image registration accuracy can be achieved. Experimental results demonstrate the effectiveness of the presented method.

1. Introduction

Motion estimation is of great importance to many image processing and computer vision applications. It plays a vital role in image mosaic, image compression, video enhancement, scene representation, and so forth. [1–6]. Numerous image registration techniques have been developed throughout the years [1, 7–9]. These methods can be loosely divided into the following groups: algorithms that use image pixel values directly, for example, correlation methods [10]; algorithms that use the frequency domain, for example, Fourier-based methods [11–14]; algorithms that use low-level features such as edges and corners, for example, feature-based methods [15]; and algorithms that use high-level features such as identified (parts of) objects or relations between features [16], for example, graph-theoretic methods [17]. Fourier-based methods, which take advantage of fast Fourier transform (FFT), are among the most popular.

The theory background of Fourier-based image registration is described in [18, 19]. Based on the shift property of Fourier transform, the translation can be robust-estimated by phase correlation. In order to account for rotations and scaling, the image is transformed into a uniform polar or log-polar Fourier representation. As a result, rotations and scalings are reduced to translations, which can be estimated by phase correlation again. It has been shown that Fourier-based methods are robust to noise and time varying illumination disturbances [20, 21].

In the Fourier-based method, the largest challenge is to evaluate the log-polar Fourier transform efficiently and accurately. FFT is commonly defined on the Cartesian grid. To calculate the log-polar coordinates coefficients, one might calculate the FFT coefficients on the Cartesian grid and then interpolate them to the log-polar grid (see [19] for more details). Unfortunately, such an interpolation process is not accurate enough and results in significant errors. Based on the technique of pseudopolar fractional Fourier transform (PPFFT) [11], a method is developed in [20] to estimate large translations, rotations, and scaling in images. It is demonstrated in [20] that a more accurate estimation of registration parameters can be achieved by PPFFT compared to the method given in [19]. However, the PPFFT method [20] can only recover scales up to 4. Recently, Pan et al. [21] present a novel method called multilayer fractional Fourier transform (MLFFT) for the fast and accurate polar/log-polar Fourier transforms. The advantages of MLFFT over other fast log-polar Fourier transform methods have been proven theoretically and experimentally in [21].

Though PPFFT and MLFFT have tremendous improvements in the fast polar/log-polar Fourier transform, they are still unsatisfying in motion estimation because they are not suitable for the log-polar Fourier transform. PPFFT pays more attention to the reduction of angular differences between the constructed grids and the uniform log-polar grids, and the radial differences have been ignored. On the other hand, MLFFT pays more attention to the reduction of radial differences between the constructed grids and the uniform log-polar grids, and the angular differences have been ignored.

To acquire better image registration accuracy, a method is proposed to estimate the log-polar coordinates coefficients using multilayer pseudopolar fractional Fourier transform (MPFFT). It encompasses PPFFT and MLFFT and provides a grid which is geometrically similar to the log-polar grid. The multilayer pseudopolar grid is dense at low coordinates coefficients and sparse at high coordinates coefficients. We compare the approximation accuracy of the log-polar Fourier transform using the PPFFT, MLFFT, and MPFFT. It is shown that accumulated distances of MPFFT are lower than those of PPFFT and MLFFT. As a result, large scalings in images can be estimated, and better image registration accuracy can be achieved. Experimental results demonstrate the effectiveness of the presented method.

The rest of the paper is organized as follows. Prior techniques related to FFT-based image registration are given in Section 2. In Section 3, MPFFT is described, and the accuracy of MPFFT is also analyzed. Experimental results are discussed in Section 4, and final conclusions are given in Section 5.

2. Basic Theory of Fourier Domain

2.1. Phase Correlation

Let 𝑓₁ and 𝑓₂ be two images that differ only by a displacement (𝑥₀,𝑦₀), that is, 𝑓₂𝑥,𝑦=𝑓₁𝑥−𝑥₀,𝑦−𝑦₀. (1) Their corresponding Fourier transforms 𝐹₁ and 𝐹₂ will be related by 𝐹₂𝜉,𝜂=𝑒^{−𝑗2𝜋(𝜉𝑥0+𝜂𝑦0)}𝐹₁𝜉,𝜂. (2) The cross-power spectrum of two images 𝑓₁ and 𝑓₂ with Fourier transforms 𝐹₁ and 𝐹₂ is defined as follows: 𝐹₁𝜉,𝜂𝐹^∗₂𝜉,𝜂𝐹₁𝜉,𝜂𝐹₂𝜉,𝜂=𝑒^{𝑗2𝜋(𝜉𝑥0+𝜂𝑦0)}, (3) where 𝐹^∗₂ is the complex conjugate of 𝐹₂; the shift theorem guarantees that the phase of the cross-power spectrum is equivalent to the phase difference between the images. By taking inverse Fourier transform of the representation in the frequency domain, we will have a function that is an impulse; it is approximately zero everywhere except at the displacement that is needed to optimally register the two images [19].

2.2. Log-Polar Coordinates Translation

Let 𝑓₂ be a translated, rotated, and scaled replica of 𝑓₁: 𝑓₂𝑥,𝑦=𝑓₁𝑠𝑥cos𝜃₀+𝑦sin𝜃₀+𝑥₀,𝑠−𝑥sin𝜃₀+𝑦cos𝜃₀+𝑦₀, (4) where 𝜃₀, 𝑠, and (𝑥₀,𝑦₀) are the rotation angle, scale factor, and translation parameters, respectively.

The Fourier transform of (4) in polar coordinate systems is 𝐼₂̂(𝑟,𝜃)=𝑒^{−𝑗(𝜔𝑥𝑥0+𝜔𝑦𝑦0)}𝑠⁻²𝐼₁̂𝑠⁻¹𝑟,𝜃+𝜃₀, (5) where 𝜔_𝑥=𝑟cos𝜃, 𝜔_𝑦=𝑟sin𝜃. Let 𝑀₁ and 𝑀₂ be the magnitudes of 𝐼₁̂ and 𝐼₂̂. Then, we have 𝑀₂(𝑟,𝜃)=𝑠⁻²𝑀₁𝑠⁻¹𝑟,𝜃+𝜃₀𝑀₂,log𝑟,𝜃=𝑠⁻²𝑀₁log𝑟−log𝑠,𝜃+𝜃₀. (6)(7)

Letting 𝜉=log𝑟,𝑑=log𝑠, (7) can be written as 𝑀₂(𝜉,𝜃)=𝑠⁻²𝑀₁𝜉−𝑑,𝜃+𝜃₀. (8)

Due to the fact that the Fourier spectrum is conjugate symmetric for real sequences, only two quadrants of the Fourier spectra of images are used to map them to the log-polar plane for estimation parameters. These parameters can be recovered by phase correlation using (8) [22].

Obviously, it is hard for us to calculate the discrete Fourier magnitudes of images in the log-polar coordinates. 2D FFT is used in traditional algorithms to approximate the log-polar Fourier representation of images. This will introduce large interpolation errors.

3. Multilayer Pseudopolar Fractional Fourier Transform

3.1. Fractional Fourier Transform

The proposed algorithm is based on the fractional Fourier transform. As in [20], the centered fractional Fourier transform is employed in the proposed method. Given a vector 𝑓=𝑓(𝑥),−𝑁/2≤𝑥≤𝑁/2, the fractional Fourier transform is defined as follows: 𝐹^𝛼(𝑘)=^𝑁/2_{𝑛=−𝑁/2}𝑓(𝑛)𝑒^{−𝑗(2𝜋𝑘/(𝑁+1))𝛼𝑛},−𝑁2𝑁≤𝑘≤2. (9)

When 𝛼=1, the fractional Fourier transform is equal to the discrete Fourier transform; thus, we get the values of the 𝑁+1 frequencies that are distributed uniformly in −𝜋≤𝜔≤𝜋 in the frequency domain.

When 0<𝛼<1, the fractional Fourier transform returns the values of the 𝑁+1 frequencies that are scattered uniformly in −𝛼𝜋≤𝜔≤𝛼𝜋 in the frequency domain.

In this paper, we only discuss the case of 𝛼=1/2^𝑚−1, where 𝑚∈𝑍⁺.

3.2. Multilayer Pseudopolar Grid

In this paper, we estimate motions of images using MPFFT. MPFFT is defined on multilayer pseudopolar grid. The set of multilayer pseudopolar grid points (MP^𝐿) is defined as follows:MP^𝐿=−𝑙𝑘2^𝑚−1𝑛,𝑘2^𝑚−1∪𝑘2^𝑚−1,−𝑙𝑘2^𝑚−1𝑛𝑛∣−2𝑛≤𝑘,𝑙≤2, (10) where 𝑛,𝐿∈𝑍⁺,𝑙,𝑘∈𝑍, and 𝑚={1,2,3,…,𝐿} (e.g., see Figure 1(d)). If 𝐿=1, this grid is the same as the pseudopolar grid. MP^𝐿 can be viewed as a composite of pseudopolar grid and multilayer grid, but it is denser than them.

(a)

(b)

(c)

(d)

Figure 1 

(a) log𝑃; (b) PP; (c) ML2; (d) MP2. The two-layer pseudopolar is much closer to the log-polar grid.

Let log𝑃 be the set of log-polar grid points, PP the set of pseudopolar grid points, and ML^𝐿 the set of 𝐿-layer Cartesian grid points: log𝑃=cos2𝜋𝑘𝑛⋅𝑛^𝑙/𝑛,sin2𝜋𝑘𝑛⋅𝑛^𝑙/𝑛,−∣0≤𝑘,𝑙≤𝑛,𝑘,𝑙∈𝑍PP=2𝑙𝑘𝑛,𝑘𝑘,−2𝑙𝑘𝑛𝑛∣−2𝑛≤𝑘,𝑙≤2,𝑘,𝑙∈𝑍,ML^𝐿=𝑘𝑚,𝑙𝑚𝑛∣−2𝑛≤𝑘,𝑙≤2,𝑘,𝑙∈𝑍, (11) where 𝑛,𝐿∈𝑍⁺,𝑚={1,2,3,…,𝐿}. Figure 1 shows examples of log𝑃, PP, and ML^𝐿. We note that if 𝐿=1, this ML^𝐿 grid is the same as the Cartesian grid.

From Figure 1, we observed that the multilayer pseudopolar grid is much closer to the log-polar grid than to the pseudopolar grid and the multilayer grid.

3.3. The Interpolation Error

In this section, the interpolation errors of PPFFT, MLFFT, and MPFFT will be discussed.

For a certain image’s spectrum magnitude 𝑀, the interpolation error is defined as follows [20]: 𝜀≈_𝑖,𝑗𝜀_Grid𝑟_𝑖,𝜃_𝑗⋅∇𝑀𝑟_𝑖,𝜃_𝑗, (12) where 𝜀_Grid(𝑟_𝑖,𝜃_𝑗) is the distance between the actual point in the frequency plane and the closest point in the grid. Because |∇𝑀(𝑟_𝑖,𝜃_𝑗)| are different for different signals, we substitute |∇𝑀(𝑟_𝑖,𝜃_𝑗)| with max{|∇𝑀(𝑟_𝑖,𝜃_𝑗)|}. It follows from (12) that 𝜀≤_𝑖,𝑗𝜀_Grid𝑟_𝑖,𝜃_𝑗⋅max∇𝑀𝑟_𝑖,𝜃_𝑗. (13) By calculating (13) with such a constant max{|∇𝑀(𝑟_𝑖,𝜃_𝑗)|}, we are able to compare the max interpolation errors of different transform methods without considering the signal itself [21].

In the rest of this paper, we only consider two-layer MP (MP²). Other MPs can be discussed similarly.

We denote 𝐶 as the set of Cartesian grid points: )𝑛𝐶=(𝑘,𝑙∣−2𝑛≤𝑘,𝑙≤2,𝑘,𝑙∈𝑍. (14) Let ML²₁ be the set of the first layer Cartesian grid points in ML²: ML²₁=𝑘2,𝑙2𝑛∣−2𝑛≤𝑘,𝑙≤2,𝑘,𝑙∈𝑍. (15) Let ML²₂ be the set of the second layer Cartesian grid points in ML². In other words, ML²₂ is the complement set of ML²₁ in ML². We denote MP²₁ as the set of first-layer pseudopolar grid points in MP²: MP²₁=−𝑙𝑘𝑛,𝑘2𝑘2,−𝑙𝑘𝑛𝑛∣−2𝑛≤𝑘,𝑙≤2,𝑘,𝑙∈𝑍. (16) Furthermore, we denote MP²₂ as the set of second-layer pseudopolar grid points in MP². In other words, MP²₂ is the complement set of MP²₁ in MP².

The distance between a point on log𝑃 and the closest point on PP can be expressed as follows [20]: 𝜀²_{Grid_PP}𝜃_i,𝑟_𝑗=𝑋_{closest_PP}𝑖,𝑗−𝑋_{real_log𝑃}𝑖,𝑗²+𝑌_{closest_PP}𝑖,𝑗−𝑌_{real_log𝑃}𝑖,𝑗², (17) where (𝑋_{closest_PP}(𝑖,𝑗),𝑌_{closest_PP}(𝑖,𝑗)) in pseudopolar grid is the closest point of (𝑋_{real_log𝑃}(𝑖,𝑗),𝑌_{real_log𝑃}(𝑖,𝑗)) in log-polar grid. In pseudopolar grid it is derived in [20] that Δ𝜃_PP𝑗≈2𝑛𝑛²+4𝑗²,Δ𝑟_PP(𝑖)=𝑖1+4𝑛²,𝑛𝑖,𝑗=1,…,2, (18) where Δ𝜃_PP is the angle between two neighboring polar radii, and the Δ𝑟_PP is the distance between two points on neighboring polar radii in pseudopolar grid. Furthermore, it is shown in [20] that 𝜀_{Grid_PP}<𝜀_{Grid_𝐶}, (19) where 𝜀_GridPP is the distance between PP and log𝑃,𝜀_{Grid_𝐶} is the distance between 𝐶 and log𝑃.

The distance between ML² and log𝑃 can be expressed as 𝜀_{Grid_ML2}=𝜀_{Grid_ML21}+𝜀_{Grid_ML22}. (20) In ML²₁, we note that Δ𝑖=Δ𝑗=1/2. Therefore, 𝜀_{Grid_ML21}<𝜀_{Grid_𝐶}𝑛𝑖,𝑗,−4𝑛≤𝑖,𝑗≤4. (21) The points in ML²₂ coincide with the points in 𝐶. It follows that 𝜀_{Grid_ML2}<𝜀_{Grid_𝐶}. (22)

The distance between MP² and log𝑃 can be expressed as 𝜀_{Grid_MP2}=𝜀_{Grid_MP21}+𝜀_{Grid_MP22}. (23) In MP² (see Figure 2), we notice that Δ𝜃_MP²=Δ𝜃_PP,Δ𝑟_MP²₂=Δ𝑟_PP,Δ𝑟_MP²₁=12Δ𝑟_PP, (24) where Δ𝜃_MP² is the angle between two neighboring polar radii in MP², Δ𝑟_MP²₂ is the distance between two points on one polar radius in MP²₂, and Δ𝑟_MP²₁ is the distance between two points on one polar radius in MP²₁. As a consequence, 𝜀_{Grid_MP2}<𝜀_{Grid_PP}. (25)

Figure 2 

Geometrical properties of the MP2.

The points in MP²₂ coincide with the points in PP, and from (19) we reach the following conclusion: 𝜀_{Grid_MP22}<𝜀_{Grid_ML22}. (26)MP²₁ is denser than ML²₁, and if we magnified the ML²₁ and MP²₁ two times, they should be the same as the 𝐶 and PP. Then, we obtain 𝜀_{Grid_MP}²₁<𝜀_{Grid_ML}²₁. (27) From (26) and (27), we get 𝜀_{Grid_MP}²<𝜀_{Grid_ML}². (28)

It follows from (19), (22), (25), and (28) that the distance between the log-polar grid points and two-layer pseudopolar grid points is the shortest one. We calculate the average distances between the log-polar grid points and other grid points. The results which are listed in Table 1 demonstrate the foregoing discussion.

Table 1 

Average distances between log-polar grid and other grids.

4. Experiments and Results

The proposed registration algorithm was tested using a large set of test images which were rotated, scaled, and translated.

4.1. A Framework for Image Registration with MPFFT

Let 𝐼₁ and 𝐼₂ be the target images. The algorithm operates as follows.(i)Input images 𝐼₁ and 𝐼₂. Apply a Blackman window function to them if it is necessary, as shown in Figure 3.(ii)Choose the proper layer (𝛼). In this paper, we use the two-layer pseudopolar grid.(iii)Calculate the MPFFT magnitudes of 𝐼₁ and 𝐼₂. We get MP^𝛼₁ and MP^𝛼₂. Then apply a high-pass filter to them.(iv)Calculate the log-polar Fourier transform magnitude spectrums of 𝐼₁ and 𝐼₂ by bilinear interpolation on the multilayer pseudopolar grid.(v)Calculate the scale factor and the rotation angle by the phase correlation technique on the log-polar Fourier transform spectrums of both images.(vi)Apply the scale factor and the rotation angle to the reference image and use the phase correlation again to detect the translation.

(a)

(b)

(c)

(d)

(e)

Figure 3 

Registration results of images. (a) Reference images. (b) Target images. (c) Registration results by PPFFT. (d) Registration results by MLFFT. (e) Registration results by MPFFT.

4.2. Registration for Arbitrary Rotation and Large Scaling Images

Registration experiments of the MPFFT algorithm were provided by comparing with PPFFT and MLFFT in this section. All the experiments were tested on the 256×256 images which were rotated 𝜃₀ and scaled 1/𝑎 as shown in Figure 3. All the test images were preprocessed by a high-pass filter to reduce the scaling overlapping [23]. Some of the test images were preprocessed by a Blackman window to decrease the rotation aliasing [24]. We presented the numeric registration results for these images in Table 2. Two-layer pseudopolar fractional Fourier transform method was shown to be more accurate than pseudopolar fractional Fourier transform method and two-layer fractional Fourier transform method by testing the Cameraman image with the Blackman window, the Lena image, and the Westconcordortho image without the Blackman window.

Table 2 

Registration results of Figure 3.

4.3. Registration for Noisy Images

All the experiments in this section were tested on the 256×256 images which were rotated and scaled as shown in Figure 4. It is well known that Fourier-based methods are robust to noise. MPFFT algorithm was verified to be also robust to noise by experiments. The images of Figure 4(b) are target images with noise. The registration results of noisy images by PPFFT, MLFFT, and MPFFT are the images in Figures 4(c), 4(d), and 4(e). We present the numeric registration results for these images in Table 3.

Table 3 

Registration results of Figure 4.

(a)

(b)

(c)

(d)

(e)

Figure 4 

Registration results of noisy images. (a) Reference images. (b) Target images with noise. (c) Registration results by PPFFT. (d) Registration results by MLFFT. (e) Registration results by MPFFT.

4.4. Registration for Real Images

The images of Figures 5(a) and 5(b) were used to test the estimation of real images. In this section, MPFFT was tested on the real images which were rotated and scaled, as shown in Figure 5, similar to previous experiments. The images of Figure 5(a) are reference images, and the images of Figure 5(b) are target images. The registration results of campus images by PPFFT, MLFFT, and MPFFT are the images in Figures 5(c), 5(d), and 5(e). It was shown from the experiments that PPFFT method could not find the rotation factor and the scale factor. MLFFT method could find the scale factor but could not find the rotation factor while all the correct factors could be found by MPFFT.

(a)

(b)

(c)

(d)

(e)

Figure 5 

Registration results of campus images. (a) Reference images. (b) Target images. (c) Registration results by PPFFT. (d) Registration results by MLFFT. (e) Registration results by MPFFT.

5. Conclusions

Based on PPFFT and MLFFT, a new image registration algorithm called MPFFT was presented for recovering large translations, rotations, and scaling between images. The multilayer pseudopolar grid was utilized to approximate log-polar grid to improve the accuracy of the PPFFT and MLFFT in the proposed algorithm. There are some problems that deserve further studies. For example, although our algorithm is more accurate than PPFFT and MLFFT, the computing cost is twice (layer number) as that of the PPFFT algorithm and twice as that of the MLFFT algorithm. Further works will focus on these problems.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 60973157, and Ming Li thanks the supports in part by the 973 plan under the Project Grant no. 2011CB302800, by the National Natural Science Foundation of China under the Project Grant Nos 61272402, 61070214, and 60873264.