Abstract
It has been shown that iterative reweighted strategies will often improve the performance of many sparse reconstruction algorithms. Iterative Framework for Sparse Reconstruction Algorithms (IFSRA) is a recently proposed method which iteratively enhances the performance of any given arbitrary sparse reconstruction algorithm. However, IFSRA assumes that the sparsity level is known. Forward-Backward Pursuit (FBP) algorithm is an iterative approach where each iteration consists of consecutive forward and backward stages. Based on the IFSRA, this paper proposes the Iterative Forward-Backward Pursuit (IFBP) algorithm, which applies the iterative reweighted strategies to FBP without the need for the sparsity level. By using an approximate iteration strategy, IFBP gradually iterates to approach the unknown signal. Finally, this paper demonstrates that IFBP significantly improves the reconstruction capability of the FBP algorithm, via simulations including recovery of random sparse signals with different nonzero coefficient distributions in addition to the recovery of a sparse image.
1. Introduction
Compressed Sensing (CS) is a new paradigm in signal processing which was put forward by [1, 2]. Many algorithms have been proposed to solve this problem, which seems to be intractable. They can be roughly divided into three categories: Greedy Pursuit, Convex Relaxation, and Bayesian Framework. Greedy methods iteratively identify elements of the estimated support set. At last, these methods use a simple least-square to recover the original signal. They mainly include Matching Pursuit (MP) algorithm [3], Orthogonal Matching Pursuit (OMP) [4], Subspace Pursuit (SP) [5], Compressive Sampling MP (CoSaMP) [6], Look Ahead OMP (LAOMP) [7], and Forward-Backward Pursuit (FBP) [8]. CS has been widely used in many fields, such as wireless sensor network [9, 10] and magnetic resonance imaging (MRI) [11–14].
In [15], the author proposes a general iterative framework to improve the performance of any arbitrary sparse reconstruction algorithm, called Iterative Framework for Sparse Reconstruction Algorithms (IFSRA). After applying the framework to MP, OMP, CoSaMP, BPDN [16], and Smoothed L0 (SL0) [17], the performance of those algorithms has been raised. However, IFSRA requires the sparsity level to be known.
FBP, a novel two-stage greedy approach, uses a forward step to enlarge the support estimate by atoms, while the backward step eliminates atoms from it. In [8], the author demonstrates that the exact recovery of FBP can be significantly better than OMP, while the run times of FBP and OMP are almost the same. However, this paper shows that the performance of FBP can be further enhanced.
By inheriting the iterative idea of IFSRA, this paper proposes the Iterative Forward-Backward Pursuit (IFBP) algorithm. Different from the IFSRA, the IFBP does not need to know the sparsity level in advance. By setting appropriate initial atoms and iterative step, IFBP gradually iterates to approach the spare signal. The simulations demonstrate that the exact recovery rate of IFBP is significantly better than FBP.
2. Compressed Sensing and Reconstruction Algorithm
2.1. Compressed Sensing Theory
Consider a standard CS measurement, where a sparse signal is collected through linear measurements viawhere is a sparse signal of length , denotes the number of nonzero elements in , is an random matrix, and is the observation vector of length with . However, it is analytically ill-posed to recover from the observation vector . Because is a sparse signal, CS reformulates (1) as a sparsity-promoting optimization problem:where indicates the number of nonzero elements in .
2.2. Iterative Framework for Sparse Signal Reconstruction
The seminal work by Candès et al. [18] shows that the reconstruction performance of Convex Relaxation Methods can be improved by a reweighted strategy. In [15], the author extends the iterative framework to an arbitrary sparse reconstruction algorithm.
Each iteration of IFSRA mainly includes three major tasks: estimation, fusion, and regularization.(i)Estimation: Use original algorithm to get the result of the regularized sparse problem.(ii)Fusion: Keep only potential atoms from the intersection of the estimate of this regularized sparse reconstruction problem and the estimate in the previous iteration.(iii)Regularization: Regularize the measurement matrix and the measurement vector which is prepared for the next iteration.
The termination condition of IFSRA is that the norm of the regularized measurement vector increases.
2.3. The Forward-Backward Pursuit Algorithm
FBP, as an iterative two-stage algorithm, needs two factors: and (). The former is called the forward step size, while the latter is called the backward step size. At the first stage, FBP expands the estimated support set by atoms. Subsequently, the algorithm computes the orthogonal projection of the observed vector onto the subspace defined by the estimated support set. At the second stage, FBP eliminates atoms with the smallest contributions to the projection to reduce the size of the estimated support set. The termination condition, which controls whether to stop these forward and backward steps, is related to the energy of the residue. If the energy of the residue either vanishes or is less than a threshold, which is proportional to the energy of the observed vector, the algorithm has to return a result. Algorithm 1 gives the pseudocode of the FBP algorithm.
| ||||||||||||||||||||||||||||||||||||||||||||||||
Compared with OMP, SP, and BP, FBP has obvious advantages, when the magnitudes of the nonzero elements start spanning a wider range, as for the Gaussian distribution or the uniform distribution. When increasing and keeping fixed, we can improve the recovery performance, while the run time also increases. From the simulations in [8], we observe that choosing and keeping result in the optimal recovery performance. Moreover, the parameter , which is useful in case of a fail, also decides the number of the iterations. In order to adapt to the IFBP algorithm, this paper make a little change to the pseudocode of the FBP algorithm in Algorithm 1. Note that, by ignoring the variable in Algorithm 1, we get the standard FBP.
3. Iterative Forward-Backward Pursuit
3.1. The Iterative Framework
By exploiting the information available in the estimate of the current iteration, the IFSRA algorithm improves the sparse reconstruction performance in the subsequent iteration(s). Assuming that the estimated support set , which is obtained in the iteration, contains true atoms, we need to identify only true atoms from atoms listed in . Based on this idea, the new problem in the iteration can be interpreted as a reduced dimensional problem. Thus, we only need to recover a sparser signal than , while the number of measurements is also . Thus, when retaining true atoms in , we are likely to obtain a better signal estimate in the iteration [15].
Because the sparsity level is known, IFSRA can easily determine the number of the reserved atoms. However, if we do not know the sparsity level , the iteration will not perform well. For example, FBP would return atoms, when FBP does not accurately reconstruct the original signal. If we retain all atoms without any further processing, the simulation shows that the iterative framework has no effect on improving the performance of FBP. This paper solves the problem by adding two parameters: and . The parameter denotes the number of the reserved atoms after the first iteration. The parameter denotes the number of the added atoms in the subsequent iteration(s). Experimental simulation results show that this iterative framework can effectively improve the performance of FBP, without the need for the sparsity level .
3.2. The Iterative Forward-Backward Pursuit Algorithm
In [8], the author points out that we can get the optimal recovery performance in practice, when choosing and . In other words, for and , when , the result of the algorithm is the best for both exact recovery rate and reconstruction error. The performance of is just lower than . This paper proposes the IFBP algorithm to further enhance the performance of FBP by using the iterative framework.
First, IFBP runs FBP. If FBP exits the iteration in the way that the energy of the residue is less than the default threshold, IFBP uses the result of the FBP as the final result. Different from IFSRA, IFBP algorithms are able to get the estimated signal after the first iteration. Note that IFSRA needs at least two iterations due to the iterative termination condition. If FBP exits the iteration in the way that the maximum size of the estimated support set is more than , IFBP retains potential atoms in the estimated support set. In the subsequent iterations, IFBP adds only potential atoms to the estimated support set after fusing the estimate of this regularized sparse reconstruction problem and the estimate in the previous iteration. IFBP is iterated as long as the energy of the regularized measurement vector decreases.
3.3. The Acceleration Strategy
Both IFSRA and IFBP improve the performance of the algorithms through the iterations. IFSRA only offers a general iterative framework. However, this paper would exploit the character of FBP to reduce the run time of IFBP.
In the () iteration, the FBP algorithm, which is the parent algorithm of IFBP in this paper, could exit the iteration in the way that the energy of the residue is less than the default threshold. Once this happens, the algorithm is likely to have found all the remaining real atoms. If we add all of those atoms to the estimated support set at once, the number of iterations will be greatly reduced. Meanwhile, the exact recovery rate of IFBP would not decline. Algorithm 2 gives the pseudocode of the IFBP algorithm.
| ||||||||||||||||||||||||||||||||||||||||||||
The main idea of the proposed algorithm is using two parameters to reconstruct the signal step by step. The proposed algorithm overcomes the problem that iterations need the sparsity level in [15]. By preserving the useful atoms in each iteration, the proposed algorithm expands the estimated support set step by step. Meanwhile, the proposed algorithm exploits the character of FBP to reduce the number of iterations. Thus, the proposed algorithm applies the reweighting scheme into the FBP algorithm without the sparsity level.
4. Experimental Evaluation
4.1. The Recovery of the Sparse Signals
In order to demonstrate the superiority of the algorithm, we compare the exact recovery rates, average recovery error, and run times of IFBP with those of FBP for signals with nonzero elements drawn from the Gaussian and uniform distributions. The nonzero entries of the Gaussian sparse signals are drawn from the standard Gaussian distribution. Meanwhile, nonzero elements of the uniform sparse signals are distributed uniformly in . The observation matrix, which is different for each test signal, is drawn from the Gaussian distribution with mean 0 and standard deviation . Then, we normalize each column norm to unity. During the experiments, we use MATLAB to perform these algorithms.
In these simulations, we choose and while varies from 15 to 48. For each , recovery simulations are repeated over 500 times, where the randomly generated Gaussian and uniform sparse signals are different for each test. We use the Average Normalized Mean-Squared-Error (ANMSE) to scale the recovery error, which is defined as where denotes the recovery of the th test vector . In addition, another evaluation criterion, called the exact recovery rate, is also used in these tests. It represents the ratio of perfectly recovered test samples to the whole test data. The exact recovery condition is selected as [8].
In these tests, we select and the termination parameter , which are the same for FBP and IFBP. Meanwhile, this paper selects and for IFBP.
Reference [8] points out that choosing and leads to the optimal recovery performance in practice. This paper chooses FBP (, ) and FBP (, ) as the parent algorithm of IFBP, respectively.
First, we test the performance of IFBP with FBP (, ) as the parent algorithm. Figures 1 and 2 depict the reconstruction performance of IFBP for the Gaussian sparse signals in comparison to FBP (, ). Analogous results are provided in Figures 3 and 4 for the uniform ensemble as well.
In Figure 1, we observe that the exact recovery rates of IFBP are significantly better than FBP. When is small, both FBP and IFBP can guarantee 100% to reconstruct the original signal. With increasing, the exact recovery rates of two algorithms decrease. At , FBP starts to fail. However, IFBP failures begin when . The exact recovery rate of FBP drops to 33.6% at , where IFBP is 61.8%. By the proposed method of this paper, the exact recovery rate of FBP at increases by 80%. As for ANMSE, IFBP is also the better performer from Figure 2. At , the ANMSE of FBP grows to 0.31, while IFBP reduces the ANMSE down to 0.19. Note that the performance of both IFBP and FBP declines when the coefficient distribution changes from the Gaussian distribution to the uniform distribution. This is related to the involved correlation-maximization step, that is, choosing the largest magnitude elements of , which becomes more prone to errors when the nonzero elements of the underlying sparse signals span a narrower range [8].
Investigating Figures 3 and 4, which depict recovery results for the uniform sparse signals, we observe a similar behavior as well. IFBP results in significant improvement over FBP in both exact recovery rates and ANMSE. The exact recovery rate of IFBP is 29.4% at = 48, which is nearly three times FBP. Meanwhile, IFBP provides 0.1 ANMSE improvement over FBP.
Then, we take FBP (, ) as the parent algorithm of IFBP in Figures 5, 6, 7, and 8. Similar to the previous test case, IFBP yields better recovery rates and ANMSE than FBP for both the Gaussian sparse signals and the uniform sparse signals.
It is noted that IFBP is computationally more demanding than the parent FBP as it runs the parent FBP multiple times to enhance the sparse signal reconstruction. In addition, it may be observed that IFBP with FBP (, ) as the parent algorithm results in a better performance, compared to IFBP with FBP (, ) as the parent algorithm. This is due to the parent algorithm, which has severe effect on IFBP.
4.2. The Recovery of the Sparse Image
In this section, we evaluate the performance of the proposed IFBP using real-world signal. We use a similar simulation setup used in [8]. The recovery is performed on the 256 × 256 image “lena,” which should be divided into 8 × 8 blocks in advance. The pretreatment is to ensure that each 8 × 8 block is sparse in the 2D Haar Wavelet basis, , where . Namely, each block only keeps the largest magnitude wavelet coefficients. Thus, the recovery problem is broken into a number of smaller and simpler problems. In order to adapt to the small size, we take the observation size , which is the same for each block. The observation matrix is randomly drawn from the Gaussian distribution with mean 0 and standard deviation . Then, we normalize each column norm to unity. The parameters are selected as and . We take FBP (, ) as the parent algorithm of IFBP in this example. Meanwhile, this example selects and for IFBP.
Figure 9 is the test image “lena.” The results of FBP and IFBP are shown in Figures 10 and 11. In this example, the Peak Signal-to-Noise Ratio (PSNR) value of FBP is 28.6 dB, while IFBP promotes the PSNR up to 29.9 dB. This example shows that IFBP has the ability to generate more accurate recovery than FBP for a signal with realistic nonzero coefficient distribution.
5. Conclusions
Combining FBP with IFSRA, this paper proposes IFBP algorithm. Unlike the IFSRA, IFBP iterates the FBP without prior information of the sparsity. IFBP can significantly improve the performance of FBP by adjusting the number of atoms which are added to the estimated support set in the first iteration and the subsequent iteration(s). Meanwhile, IFBP judges the result of the parent algorithm in the first iteration and overcomes the problem that IFSRA has to iterate at least twice due to the iterative termination condition. In addition, this paper uses the acceleration strategy to reduce the number of the iterations. The efficacy of IFBP in applications is shown using numerical simulations on both spare signals and the spare image. More important, the IFBP algorithm provides a useful idea for algorithm improvement, where the sparsity level is unknown.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the Specialized Research Fund for the Doctoral Program of Higher Education (no. 20130031110032), the National Natural Science Foundation of China (no. 61171140), and Tianjin Key Technology Program of the Ministry of Science and Technology (no. 14ZCZDNC00014).