Mathematical Problems in Engineering

Volume 2016, Article ID 6212674, 11 pages

http://dx.doi.org/10.1155/2016/6212674

## On a Gradient-Based Algorithm for Sparse Signal Reconstruction in the Signal/Measurements Domain

University of Montenegro, 81000 Podgorica, Montenegro

Received 16 March 2016; Accepted 24 May 2016

Academic Editor: Cornel Ioana

Copyright © 2016 Ljubiša Stanković and Miloš Daković. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Sparse signals can be recovered from a reduced set of samples by using compressive sensing algorithms. In common compressive sensing methods the signal is recovered in the sparsity domain. A method for the reconstruction of sparse signals which reconstructs the missing/unavailable samples/measurements is recently proposed. This method can be efficiently used in signal processing applications where a complete set of signal samples exists. The missing samples are considered as the minimization variables, while the available samples are fixed. Reconstruction of the unavailable signal samples/measurements is preformed using a gradient-based algorithm in the time domain, with an adaptive step. Performance of this algorithm with respect to the step-size and convergence are analyzed and a criterion for the step-size adaptation is proposed in this paper. The step adaptation is based on the gradient direction angles. Illustrative examples and statistical study are presented. Computational efficiency of this algorithm is compared with other two commonly used gradient algorithms that reconstruct signal in the sparsity domain. Uniqueness of the recovered signal is checked using a recently introduced theorem. The algorithm application to the reconstruction of highly corrupted images is presented as well.

#### 1. Introduction

A signal is sparse in a transformation domain if the number of nonzero coefficients is much lower than the number of signal samples. For linear signal transforms the signal samples can be considered as linear combinations (measurements) of the signal transform coefficients. Sparse signals can be fully recovered from a reduced set of samples/measurements. Compressive sensing theory is dealing with sparse signal reconstruction [1–22]. In common signal processing problems, for signals sparse in a transformation domain, signal samples can be considered as observations since they are linear combinations of the transformation domain coefficients. The goal of compressive sensing is to reduce the set of signal measurements while preserving complete information about the whole signal. In some applications unavailable signal samples result from the system physical constraints including unavailability to perform all signal measurements. Signals may also contain arbitrarily positioned samples that are heavily corrupted. In these cases it is better to remove such samples and consider them as unavailable in the signal recovery [6]. Compressive sensing theory based reconstruction algorithms may be also used when the unavailable signal samples are not a result of an intentional strategy to compress the data. Note that one of the initial compressive sensing theory successes in applications (computed tomography reconstruction) was not related to the intentional compressive sensing strategy but to the physical problem constraints, restricting the set and positions of available data.

Various algorithms for reconstruction of sparse signals from a reduced set of observations are introduced [8–17]. Two of the most important approaches are based either on the gradient of the sparsity measure [8] or on the orthogonal matching pursuit methods [16].

The topic of this paper is the reconstruction of signals with some arbitrary positioned unavailable samples. These samples may result from a compressive sensing strategy or from their unavailability due to various reasons. A method for the unavailable signal samples reconstruction, considering them as variables, has been proposed in [22]. In contrast to the other common methods that recover sparse signals in their sparsity domain this method reconstructs missing samples/measurements to make the set of samples/measurements complete. The available samples are fixed (invariant) during the reconstruction process. An analysis of the algorithm performance is presented in this paper. A relation between reconstructed signal bias and the algorithm step-size is derived and statistically confirmed. A new criterion for the algorithm step-size adaptation is proposed. The improved version of this algorithm is compared with two other common gradient-based algorithms proving its accuracy and computational efficiency. The discrete Fourier transform domain is used as a case study, although the algorithm application is not restricted to this transform [23]. The solution uniqueness is checked after signal reconstruction by using the recently proposed theorem [24].

The paper is organized as follows. After the definitions in the next section, the adaptive gradient algorithm is presented. In the comments to the algorithm the step-size and bias are considered and illustrated. A new criterion for the algorithm parameter adaptation is proposed in Section 3 as well. Reconstruction is illustrated on examples and uniqueness of the obtained solution is analyzed. Statistical performance are checked in Section 4. An illustration of the algorithm application to the image reconstruction is given as well.

#### 2. Gradient-Based Reconstruction of Missing Samples

Discrete-time signal with samples will be considered (with vector notation ). Its transform coefficients will be denoted by . If the number of nonzero coefficients for is such that then the signal is sparse in this transformation domain. Next assume that only signal samples are available in the time domain at the discrete-time positions:Under some conditions, studied within the compressive sensing theory [1–3], full signal reconstruction is possible based on the reduced set of available samples if the signal is sparse in a transformation domain. In the signal theory this problem can be formulated as a full recovery of the unavailable samples as well. In general, a reconstruction algorithm is based on the sparsity measure minimization. Nonzero transform coefficients’ counting is the simplest sparsity measure. Counting can be achieved by the mathematical form , referred to as the “-norm” [1, 20]. The signal reconstruction problem statement is thenVector contains the available signal samples, while the transform vector elements are the unknown transform coefficients. Matrix is the measurement matrix. In common signal transforms it corresponds to the inverse transformation matrix where the rows corresponding to the unavailable samples are omitted. If then the matrix is a submatrix of containing its rows corresponding to the discrete-time instants (1).

Although there are some approaches to reconstruct the signal using the -norm based formulation [25] in principle, this is an NP-hard combinatorial optimization problem. It is the reason why the closest convex -norm of the signal transform is used instead of the “-norm” in the minimization process:where . Under the conditions defined using the restricted isometry property (RIP) [3, 4], minimization of the -norm can produce the same result as (2) [1, 22, 26]. Common gradient-based algorithms reformulate the problem defined by (3) into a form suitable for application of standard minimization tools. Minimization is done in the sparsity domain. One such a method is briefly reviewed in Appendix.

##### 2.1. Algorithm

A gradient-based algorithm that minimizes the sparsity measure by varying the missing sample values is presented next. In this gradient-based minimization approach the missing samples are considered as variables [22]. The missing sample values are varied in an iterative way to produce lower sparsity measure values and finally to approach the minimum of the convex -norm based sparsity measure (3) with an acceptable accuracy. Note that if the reconstruction conditions are met for the -norm [3] then the -norm minimum position will correspond to the true values of the missing samples.

The signalwill be used as the initial estimate. Note that this signal would follow as a result of the -norm based minimization of the signal transform. The set of missing sample positions is . Values of the available samples are fixed and will be considered as constants. The missing sample values are varied through iterations. If we denote by the vector of signal values reconstructed after iterations, then the goal is to achieve where the components of are . In order to find the position of function minimum, the relation for iterative missing samples (variables) calculation is defined by using the gradient-based descend on the sparsity measure asVector contains the minimization variables (missing signal sample values) in the th iteration. The gradient vector coordinates can be estimated, in the th iteration, by using the finite differences of the sparsity measure. They are calculated for each missing sample at instants : where Again for the available signal values are not changed, . All gradient vector values, including the gradient zero and nonzero coordinates, in the th iteration are denoted by .

Through statistical study [22] it has been concluded that an appropriate step parameter value in (6) is related to the finite difference step as .

#### 3. Comments on the Algorithm

(i)The algorithm inputs are the signal length , the set of available samples , and the available signal values , .(ii)For the DFT instead of using signals defined by (9) and their transforms for each it is possible to calculate with and . Note that values are not dependent on the signal and the iteration number . They can be calculated only once. Similar simplification can be done for any linear signal transform .

##### 3.1. Adaptive Step

The influence of step-size to the solution (minimum position) precision is analyzed next. Assume that we have obtained the exact solution for the missing samples. The change of sparsity measure is tested by the change of signal sample for . For a signal of sparsity in the DFT domain, whose form is , the signals with the steps used for the gradient estimate calculation are and . Their sparsity measures are In the worst case, amplitudes are in phase with and when It means that . This is an important fact leading to the conclusion that the stationary state of the iterative algorithm is biased. The algorithm moves the solution to a biased value in order to produce in the algorithm stationary point. For the zero value of the gradient we have and . By replacing with in and with in the worst case sparsity measures are and . In this case the bias value can be obtained from asThe bias limit is proportional to the step-size . Obviously, the bias limit can be reduced by using small values of the step . Calculation with a very small step would be inefficient and time consuming with an extremely large number of iterations. An efficient way to use the gradient-based algorithm is in adapting its step-size. In the initial iteration the step-size of an order of signal amplitude is used:When the algorithm reaches a stationary point with this step then the bias will dominate and the mean squared error will be almost constant. The algorithm can improve the solution precision by reducing the step-size.

This simple relation between the bias and step-size has been confirmed through statistical analysis on more complex cases, with a large number of missing samples.

##### 3.2. Adaptation Criterion

The gradient-based algorithm convergence improvement by changing steps is analyzed in detail in [22]. A criterion that can efficiently detect the event that the algorithm has reached the stationary point with regard to the mean square error (the vicinity of the sparsity measure minimum defined by the bias) described in the previous subsection is proposed next. The presented criterion is based on the direction change of the gradient vector. When the vicinity of the minimum sparsity measure point is reached within the region defined by the bias value, then the gradient estimate oscillates and changes its direction for almost degrees. The angle between two successive gradient vector estimations and denoted as can be calculated as If this angle is, for example, above then we can conclude that the signal missing samples (variables) reached oscillatory nature around the minimal sparsity measure position. When these values of the angles are detected the algorithm step is reduced, for example, for or . The iterations are continued starting from the reconstructed signal values reached in the previous step.

When the minimum of the sparsity measure is reached with a sufficiently small , the value of can also be used as an indicator of the solution precision. For example, if the signal amplitudes are of order and taking in the first iteration will produce the solution with a precision better than 20 [dB]. Then if the step is reduced to a precision better than 40 [dB] will be obtained and so on. Value of can be used as the algorithm stopping criterion.

##### 3.3. Stopping Criterion

The precision of the result in iterative algorithms is commonly estimated based on the change of the result values in the last iterations. Therefore an average of changes in a large number of variables is a good estimate of the achieved precision. The value ofcan be used as a rough estimate of the reconstruction error to signal ratio. In this relation is the signal reconstructed before reduction and is the reconstructed signal after iterations done with the reduced step. Value of can be used as the algorithm stopping criterion. If the value of is above the required precision (e.g., if dB), the calculation procedure should be continued with reduced values of step .

##### 3.4. Large Step Influence

A possible divergence of a gradient-based algorithm is related to large steps . We can write In general for a complex number , with and a large , from the problem geometry we can show that the following bounds hold . Therefore, Lower limit is obtained if is imaginary-valued, while the upper limit follows if is real-valued. If is large the missing signal values will be adapted for a value independent on . The missing sample values will oscillate within the range of the original signal values until the step is reduced in iterations. The variable values will be arbitrary changed within the signal amplitude order as far as is too large. It will not influence further convergence of the algorithm, when the step assumes appropriately small values.

A pseudocode of this algorithm is presented in Algorithm 1.