Abstract
To reduce the computation complexity of wavelet transform, this paper presents a novel approach to be implemented. It consists of two key techniques: (1) fast number theoretic transform(FNTT) In the FNTT, linear convolution is replaced by the circular one. It can speed up the computation of 2D discrete wavelet transform. (2) In two-dimensional overlap-save method directly calculating the FNTT to the whole input sequence may meet two difficulties; namely, a big modulo obstructs the effective implementation of the FNTT and a long input sequence slows the computation of the FNTT down. To fight with such deficiencies, a new technique which is referred to as 2D overlap-save method is developed. Experiments have been conducted. The fast number theoretic transform and 2D overlap-method have been used to implement the dyadic wavelet transform and applied to contour extraction in pattern recognition.
1. Introduction
Wavelet transform plays an important role in image processing, pattern recognition, document analyses, and so forth [1–12]. The basic operation of it, in fact, is a linear convolution.
In digital signal processing, a signal is represented by a discrete sequence. Therefore, the discrete wavelet transform can be utilized to process it. We have [13] where Explicitly, either continuous wavelet transform or discrete wavelet transform is essentially the operation of filtering. Directly calculating (1) may be time-consuming, because the number of multiplication operations may come large in this way. Although Mallat algorithm [14] can implement the wavelet transform successfully in some special cases, that is, in an orthogonal wavelet basis or in the multiresolution analysis, it is not a solution of all kinds of wavelet transforms. Therefore, studying how to speed up the general discrete wavelet transform is of great practical significance.
Actually, the filtering is a linear convolution in signal processing. In the following, without loss of generality, we will begin with the definition of Linear Convolution of two 2D sequences.
The equation for a 2D finite impulse response of a filter can be represented in the similar way and given by where and stand for the input and output of the filter, respectively, and denote the filtering coefficients (, ).
In many certain applications, we are required to compute only such an output sequence which is with a finite length in (3). Let be a 2D finite input of the filter with lengths and . We compute the finite output as follows: In practice, the difference between the length of the input signal and that of the filter is often large. If we calculate convolution (4) directly, a large number of multiplication operations will be executed. In the meantime, a series of calculations to treat zeros will be performed, if fast Fourier transform (FFT) is used to speed up the computation. Both of these two methods will be time-consuming to compute the linear convolution. To overcome this problem, in this paper we will present a novel approach which is referred to our previous work as number theoretic transform (NTT) [15–17]. Also, we will prove that the computation of the linear convolution can be replaced by that of the cyclic convolution of two 2D sequences with lengths and , respectively.
For two 2D sequences and of finite lengths and , their cyclic convolution can be written by where denotes the remainder of modulo ; that is, , is any integer and .
The number theoretic transform (NTT) provides an effective way to calculate the cyclic convolution. However, there exist two difficulties, if directly applying the NTT to the whole input sequence, namely:(i)a considerable big modulo M has to be imposed. It obstructs the effective implementation of the NTT due to the limited length of the word which is used to store data in computers. It will, thus, become a bottleneck in the computation of convolution;(iii)the advantage of the fast computation of the NTT cannot be reached when the difference between the length of the input sequence and that of the filter sequence is large. Thus, it may not speed up the calculation of convolution in this case.
To avoid using a big modulo M, we can use Chinese remainder theorem to reduce the length of the modulo M and successfully apply the NTT to calculate the convolution. Unfortunately, the number of multiplication operations will be at least doubled, and thus the computational complexity will increase. The second key technique will be worked out in this paper to overcome the above difficulties and speed up the calculation of the convolution. It is termed 2D overlap-save method, which is an expansion of the 1D overlap-save method. It can be used to implement the 2D wavelet transform when a big difference between the length of the input sequence and that of the filter sequence occurs. This method consists of three steps.(i)First, the original 2D input sequence is equivalently divided into many small separated sections. This division brings two evident improvements.(a)The size of each section is much smaller than that of the whole input sequence, such that a small modulo M can be used, and the NTT can, therefore, be performed in a computer system.(b)The difference between the length of one section and that of the filter sequence becomes smaller, which makes the effective application of the NTT.(ii)In the second step, we calculate the cyclic convolution of each section with the filter sequence by NTT.(iii)Finally, the result of (4) is obtained by picking out data from each of the results calculated in the second step and combining these data together.
In comparison with direct method and fast Fourier transform (FFT), it will be proved explicitly that the number of multiplication operations in the 2D overlap-save method will be smaller than that in any of those two methods in many particular cases.
In the next section, the number theoretic transform (NTT) will be presented. Details of the 2D overlap-method will be discussed in Section 3. A comparison of computation of the multiplication in three methods will be given in Section 4. A computational example will be presented as well as a practical experiment in Section 5 which will verify the efficiency of the proposed approach. In the experiment, the fast number theoretic transform and 2D overlap-method were applied to implement the dyadic wavelet transform extracting the contours in recognition of Chinese handwriting.
2. Number Theoretic Transform (NTT)
Suppose that are the filter coefficients , ; and , are the input and output of the filter, respectively. In the concrete applications of the discrete wavelet transform (DWT), the lengths of the input and output are very long ( and are very big), while lengths of the filter coefficients are short ( and are small). Therefore, if the direct method is used to calculate their linear convolution, the number of multiplication operations will be large. In this section, the 2D number theoretic transform (FNTT) will be applied to calculate the 2D circular convolution of 2D sequences of lengths and . In a 2D number theoretic transform (NTT), if a modulo is too big, the Chinese remainder theorem (CRT) will be utilized to reduce the length.
2.1. Linear Convolution and 2D Circular Convolution
Given two 2-dimensional sequences and , their linear convolution is defined by Given two 2-dimensional sequences and of lengths and , respectively, their circular convolution is where denotes the remainder , .
Proposition 1. The linear convolution (6) can be computed by computing circular convolution of two 2-dimensional sequences of lengths and , respectively.
2.2. Using 2D Number Theoretic Transform to Calculate the 2D Circular Convolution
Let be a positive integer and suppose that and are two 2-dimensional integer sequences .
Let where , and and are two given integers, and denote the circular convolution as shown in (7), .
If a 2-dimensional transform (8) has an inverse transform and has circular convolution property, then congruence equation (8) is referred to as a 2-dimensional number theoretic transform and can be abbreviated to NTT .
Agarwal and Burrus gave some sufficient conditions for existence of 2-dimensional NTT given with , , and , (see [15]). In [17], the authors gave some sufficient and necessary conditions for existence. We have the following theorem.
Theorem 2. If , , , are distinct primes, then the congruence equation (8) is a 2-dimensional NTT with the first dimension of length and the second dimension of length , if and only if , and , , .
Corollary 3. The congruence equation (8) is a 2-dimensional NTT if and only if where denotes the least common multiple of two integers and and denotes the greatest common divisor of two integers and .
Corollary 4. Let be an odd prime, and . Then the congruence equation (8) is a 2-dimensional NTT , if and only if and , , and .
Clearly, from congruence equations (8), (9), (11), and (12), we can use the 2-dimensional NTT to calculate a 2-dimensional circular convolution.
Finally, we obtain
Select so that where , and is an odd.
If we use the 2-dimensional NTT to calculate the 2-dimensional circular convolution so that , then form congruence equation (14) we deduce that
2.3. Fast Number Theoretic Transform (FNTT)
The idea of FFT can be used to perform the NTT. In this subsection, a theoretic description will be presented briefly. More details will be given in Section 5.
Let
From congruence equation (17), we deduce that
Suppose that and satisfy inequalities, and we then can use the idea of FFT to calculate the congruence equations (17) and (18).
For computing every in congruence equation (17), the numbers of all multiplication necessary are by using FFT algorithm. Hence, if using FFT algorithm to calculate the congruence in congruence equation (17), the numbers of all multiplication necessary are .
Similarly, if using FFT algorithm to calculate all congruence equations in (18), then the numbers of all multiplication necessary are .
Therefore, if using the fast number theoretic transform (FNTT) to calculate the congruence equation (8), the number of all multiplication necessary is If is very large, we may reduce the length of a word by using Chinese remainder theorem and it can be abbreviated to CRT.
Therefore, we have the following proposition.
Proposition 5. Suppose that congruence equation (8) is a 2-dimensional NTT , where are distinct primes; then, we have a total of s 2-dimensional NTT , and they are described as follows:
where , ,.
If
then
where .
Proof. Because the congruence equation (8) is a 2-dimensional NTT , hence
so
Hence, the congruence equation (22) has a total of 2-dimensional NTTs . From CRT and the congruence equation (23), we deduce that the congruence equation (24) holds.
3. Two-Dimensional Overlap-Save Method
Select two positive integers and , so that and ; either or is given by an integer power of 2; that is, and where and are two positive integers.
Assuming that the input sequence , is divided into many small sections, which are termed submatrices, The consecutive matrix is overlapped by the previous one. As a simple example, four blocks chosen from the divided input matrices are overlapped and are graphically shown in Figure 1. Parameters and can be considered as shift parameters, because changing either or will select different submatrix. For the first submatrix shown in Figure 1(a), we select and . Second submatrix ( and ) is displayed in Figure 1(b). Third one with parameters of and is given in Figure 1(c). Fourth matrix shown in Figure 1(d) has parameters and . The procedure of the overlapping is also graphically illustrated in Figure 1.
(a)
(b)
(c)
(d)
For the operation of circular convolution, it is necessary to have two sequences with the same length. Taking (7) for example, the sequences and are of the same length of . Since the input sequence has already divided into submatrices of by (27), the filter sequence has to be of length . Therefore, augmenting zeros are required to be added to the row and zeros to the column of the filter sequence . As a result, the sequence becomes where and .
The following form shows how a sequence is yielded by adding augmenting zeros to the filter sequence : For each of the divided submatrices, we can give and and compute the circular convolution of two 2D sequences and , by
From (28), we deduce that The result of (31) is that of circular convolution. As we will see below, there exists an important connection between the circular convolution and the linear one. In fact, since and , we can choose and in (31); this makes and in (31). Thus, the remainder can exactly be equal to and . Then, the circular convolution of (31) becomes In order to clearly understand the procedure of obtaining the components of linear convolution from the circular convolution equation (31), a graphic description of it can be found in Figure 2. We show a dashed rectangle with size of by , in Figure 2(a), to represent the circular convolution which is described in (31) and a darkened rectangle with size of by , to indicate the linear convolution stated in (32). Obviously, if the first data in rows and the first data in columns are discarded from the dashed rectangle, the darkened rectangle can be obtained. In the following, we will explain how to obtain the result of (4).
(a)
(b)
The output of the whole equation (4) can be represented in the following matrix: which is an matrix.
So, (32) gives a submatrix of matrix (note that and that ): Next, to obtain the whole result, we need to compute the circular convolution of two 2D sequences and , . With the same reasoning as above, a consecutive submatrix of the matrix (34) is evinced as follows: The corresponding graphic description is indicated in Figure 2(b). We will darken the rectangle in Figure 2(b) to express the submatrix (35) and the dashed one on its upper-left-corner to be the graphic description of the submatrix (34).
Clearly, if , then , where and. Let represent the largest integer that is less than or equal to the real number ; therefore, when and , we will obtain all submatrices of the matrix . If and , then and . The input matrix will be divided with no remainder. Figure 3 displays a series of small rectangles. Each rectangle stands for one submatrix. The entire matrix can be obtained by combining them.
In Figure 3, there is a white -type area indicating that there are no output data. Two calculation methods can be applied to obtain the data. The first way is by using a direct method because only a few computations are needed. Alternatively, we can compute the data of -type area with the overlap-save method proposed above, if and . The detail of it is presented in the Technical Report [18]. The entire result is completed and is demonstrated in Figure 4 graphically.
The two-dimensional overlap-save method is summarized as follows.(i)Select two positive integers and , so that and . Either or is given by an integer power of 2 for using an FFT; that is, and where and are two positive integers.(ii)Let satisfy (28).(iii)Compute the circular convolution of two sequences and , . When , and , , we obtain all submatrices of the matrix of whose form is as (34). If and , then and . The input matrix will be divided with no remainder.(iv)For , , and, compute by using the direct method. If and , we will add zeros to the sequence so that the 2D sequences are of lengths and . Then, we will compute , , and , by using an overlap-save method. The detail of it is presented in the Technical Report [18].
We will introduce the way of using FNTT and 2D overlap-save method with two examples. One is numerical computation and the other is an experiment of extracting the contour of a Chinese handwritten character.
4. Comparison of the Numbers of Multiplication
Let , , , ,, , and , . Suppose that and satisfy and . Let , , and denote the numbers of multiplication necessary for computing the convolution (4) by using direct method, FNTT, or FFT to calculate (4) (using Proposition 1) and 2D overlap-save method, respectively. In [18], we compared the number of multiplications necessary for computing the convolution (4) of two 2D sequences and , by three different methods.
We proved the following results:(1)If , then Also, if , , then (2)Consider (3)If , , , and , where , then
In this section, we will suppose that . Let , , and . For and , we computer by using the direct method.
We have the following proposition.
Proposition 6. If , then
where denotes the number of multiplication necessary for computing by using the direct method, , , and , .
Also, if and , then
Proof. Let I, II, and III denote the number of multiplication necessary for computing the , , and by direct method, respectively, where Then So, Clearly, if and , then (41) holds. This completes the proof.
Proposition 7. If , , , and , where , , , then where denotes the number of multiplication necessary for computing (4) by using 2D overlap-save method.
Proof. Because , , , and , by (41), we have
Compute the cyclic convolution of two sequences and , ; that is, compute matrix (35), by using FNNT. It is well known that using FNTT to calculate the (35) requires
where .
When , , , and , then
where is a positive integer.
So, the input matrix will be divided with no remainder. When and , we obtain all matrices as (35). Thus,
equal to (45) holds. This completes the proof.
Example 8. Suppose that ; that is, , , , and ; then, If , , , and , then This example shows that in some particular cases.
5. Experiments
The theoretic description of the 2D overlap-save method has been provided in the preceding section. This section further examines how such a technique will be performed and how it will be applied to the wavelet transform. Particularly, we will use image processing as an example. The experiment will focus on the application of the 2D overlap-save method to the dyadic wavelet transform extracting the contours of Chinese handwriting.
5.1. Numerical Computation
Let denote a digital filter and let a two-dimensional sequence be the coefficients of the filter. A two-dimensional sequence represents the input of the filter. In this example, because and , the length of input matrix is and that of the filter is .
Using the overlap-save method to calculate the output of the filter means that the elements of the matrix need to be computed.
The solution is given below.
Since , , by selecting , and , we have . In order to use the FNTT, the input matrix is divided into 9 submatrices of size . Each consecutive submatrix is partly overlapped with the size of pixels. In addition, zeros of size are added to the filtering sequence to ensure that its length is equivalent to that of the submatrix. Therefore, the filtering sequence is extended to ; that is, where .
Next, we compute the circular convolution of the two sequences and , , where .
When , we arrive at the following result: After choosing all the parameters of , , all submatrices of the matrix , defined in (34), will be obtained as follows: For , , , , we can compute it by using an overlap-save method. The detail of it is presented in the Technical Report [18].
The fast number theoretic transform (FNTT) is now applied to the implementation of the circular convolution The computation is performed in the following five steps shown in Figure 5.
Step 1 (choosing ). Since
we can select so that , where 257 is a prime. Clearly, , and , for .
Thus,
is a 2D number theoretic transform. It has the following inverse transform:
where denotes an integer so that .
Step 2 (calculating the number theoretic transform of input matrix by the FNTT). For computational convenience, here, one writes
Then, (62) becomes
Use the FNTT to calculate (64) and (65).
So,
Step 3 (calculating the number theoretic transform of the filter by the FNTT). The FNTT of the filter is written as Similarly, we have
Step 4 (compute the number theoretic transform of the output of the filter). From the preceding three steps, we can compute the circular convolution of the filter by defining the product of and as where and the , is the circular convolution of (60).
Step 5 (calculate the output of the linear convolution). From (63), we deduce the inverse transform of (70): Using FNTT to calculate (71), we have Then, the linear convolution in (58) is obtained:
5.2. Application to Image Processing
The proposed method can be applied to many fields such as image processing and pattern recognition. An example of the application to the image processing is illustrated in Figure 6. The original image shown in Figure 6(a) is a 2-dimensional image of an aircraft with a size of 256 256 pixels; that is, . Our task is to extract its contours. The discrete wavelet transform with scale where was applied and the spline wavelet, described in (1), of size was chosen; that is . We use the filtering coefficients tabulated in Table 1. Using the 2D overlap-save method, this original image was divided into 25 sections, with each one having the dimension of 64 64 pixels; that is, we select , . (See Example 8 in Section 4.) Figure 6(b) shows these subimages. The overlapped part can be explicitly observed in each subimage.
(a)
(b)
To facilitate the presentation of the 2D overlap-save method, the procedure of it is diagonally displayed in Figure 7, where a series of the divided subimages are illustrated. The image in Figure 6(a) was equivalently divided into 25 sections by applying (27). In Figure 7(a), we represent 5 subimages of , , , , , and , , and , , consecutively. The lower-right corner of each subimage is repeated in the upper-left corner of its consecutive subimage. Four subimages are displayed in Figure 7(b), and the parameters of , , , , , , and , were chosen. In Figure 7(c), we selected 3 subimages of , , , , and , . There are 3 subimages of , and , in Figure 7(d). Figure 7(e) displays subimages of , . In Figure 7(f), , , , , , , and , were chosen. In Figure 7(g) the parameters were decided upon , , , , and , . Two subimages are shown in Figure 7(h) by selecting , , and , . There is one subimage of , in Figure 7(i).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
We select a 2D NTT which has inverse transform where denotes integer so that , and .
The FNTT was applied to calculate the circular convolution of each subimage. After applying (31) to each of the subblocks, a series of segments of the contours were obtained.
By using (32), we can gradually reach the final result of the linear convolution. Figure 8(a) illustrates those pixels which satisfy (35), where , , , and . Combining them together, the contours of the whole character were obtained and displayed in Figure 8(c) which excludes the -type border. Next, the -type border was computed by direct method. Figure 8(b) shows the result. Combining this result with that shown in Figure 8(c), the entire contours of the Chinese handwriting were combined. Figure 8(d) represents the entire contour.
(a)
(b)
(c)
(d)
6. Conclusions
A novel approach to reduce the computation complexity of the wavelet transform has been presented in this paper. Two key techniques have been applied in this new method, namely, fast number theoretic transform (FNTT) and two-dimensional overlap-save technique. In the fast number theoretic transform, the linear convolution is replaced by the circular convolution. It can speed up the computation of 2D discrete wavelet transform. Directly calculating the fast number theoretic transform to the whole input sequence may meet two difficulties; namely, a big modulo obstructs the effective implementation of the fast number theoretic transform and a long input sequence slows the computation of the fast number theoretic transform down. To fight with such deficiencies, a new technique which is referred to as 2D overlap-save method has been developed. Experiments have been conducted. The fast number theoretic transform and 2D overlap-method have been used to implement the dyadic wavelet transform and applied to contour extraction in image processing and pattern recognition.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the Research Grants MYRG205(Y1-L4)-FST11-TYY and MYRG187(Y1-L3)-FST11-TYY and Chair Prof. Grants RDG009/FST-TYY of University of Macau, as well as Macau FDC Grants T-100-2012-A3 and 026–2013-A. This research project was also supported by the National Natural Science Foundation of China 61273244.