Abstract
We present a probabilistic analysis on conditions of the exact recovery of block-sparse signals whose nonzero elements appear in fixed blocks. We mainly derive a simple lower bound on the necessary number of Gaussian measurements for exact recovery of such block-sparse signals via the mixed norm minimization method. In addition, we present numerical examples to partially support the correctness of the theoretical results. The obtained results extend those known for the standard minimization and the mixed minimization methods to the mixed minimization method in the context of block-sparse signal recovery.
1. Introduction and Main Results
The problem of block-sparse signal recovery naturally arises in a number of genetics, image processing, and machine learning tasks. Prominent examples include DNA microarrays [1], wavelet sparsity modeling [2], color imaging [3], and wideband spectrum sensing [4]. In these contexts, we often require to recover an unknown signal from an underdetermined system of linear equations , where are available measurements and is a measurement matrix. Unlike previous works in compressed sensing (CS) [5–7], the unknown signal not only is sparse but also exhibits additional structure in the form where the nonzero coefficients appear in some fixed blocks. We refer to such a structured sparse vector as block-sparse signal in this paper. Following [8–11], we only consider the nonoverlapping case in the present study. Thus, from mathematical point, a block signal can be viewed as concatenation of in blocks of length ; that is, where denotes the th block of and (). In these terms, we say that is block -sparse if has nonzero Euclidean norm for at most blocks. In this paper, we furthermore assume that each element in these nonzero blocks has nonzero coefficient. Obviously, if , the block-sparse signal degenerates to the conventional sparse signal well studied in compressed sensing.
Denote
where is an indicator function; that is, , if ; , otherwise. Thus a block -sparse signal can be defined as a signal that satisfies . It is known that, under certain conditions on measurement matrix (i.e., [8]), there is a unique block-sparse signal that obeys the observation and can be exactly recovered by solving the problem
Similar to the standard minimization problem, (3) is NP-hard and computationally intractable except for very small size. Motivated by the study of CS, one then commonly uses the strategy to replace the norm with its closest convex surrogate norm, thus to solve a mixed norm minimization problem: where . This model can be treated as a second-order cone program (SOCP) problem and many standard software packages can be used for the solutions very efficiently. In many practical cases, the measurements are corrupted by bounded noise; then we can apply the modified SOCP or the group version of basis pursuit denoising (BPDN, [12]) program as the following: where is a tuning parameter, which controls the tolerance of the noise term. There are also many methods to solve this optimization problem efficiently, such as the block-coordinate descent technique [13] and the Landweber iterations technique [14]. Conditions under which solving problem (4) can successfully recover a block-sparse signal have been extensively studied [8, 10, 15]. For example, Eldar and Mishali [8] generalized the conventional RIP notion to the block-sparse setting and showed that if satisfies the block-RIP (see Definition 1) with constant , solving (4) can accurately get any block-sparse solution of .
Among the latest researches in compressed sensing, many authors [16–21] have showed that minimization with allows the exact recovery of conventional sparse signals from much fewer linear measurements than that by minimization. Naturally, it would be interesting to make an ongoing effort to extend minimization to the setting of block-sparsity. Specifically, the following mixed norm minimization problem is proposed (see [11])
for block-sparse signal recovery, as a generalization of the standard minimization, where . Similar to the standard minimization problem, (6) is also a nonconvex problem for any , and finding its global minimizer is in general computationally impossible. However, it is well known that there are several efficient heuristic methods to compute local minimizers of the standard minimization problem; say, for example, [18, 22]. One can generalize those approaches to solve the mixed minimization problem (6). In particular, we will adopt the iteratively reweighted least squares techniques in this paper.
In [17], Chartrand and Staneva conducted a detailed analysis of the minimization approach for the nonblock sparse recovery problem. They derived a lower bound of Gaussian measurement for exact recovery of a nonblock sparse signal. Furthermore, Eldar and Mishali [8] also provided a lower bound on block-sparse signal recovery in the Gaussian measurement ensemble for the mixed minimization. Along this line, we will provide in this paper a lower bound of Gaussian measurements for exact recovery of block-sparse signal through the mixed norm minimization. The obtained results will complement the results of [8, 17] and demonstrate particularly that the block version of minimization can reduce the number of Gaussian measurements necessary for the exact recovery as decreases.
To introduce our results, we first state the definition of block-RIP [8] as follows.
Definition 1 (see [8]). Let be an measurement matrix. One says that has the block-RIP over (then ) with constant if for every block -sparse signal over such that
This new definition of block-RIP is crucial for our analysis of the mixed minimization method. For convenience, we still use instead of , henceforth to represent the block-RIP constant of order . With this notion, we can prove the following sufficient condition for exact recovery of a block -sparse signal.
Theorem 2. Let be an arbitrary block -sparse signal. For any given , if the measurement matrix has the block-RIP (7) with constant
then the mixed minimization problem (6) has a unique solution, given by the original signal .
Condition (8) generalizes the condition of the mixed case in [8] to the mixed case. Note that the right-hand side of (8) is monotonically decreasing with , which shows that decreasing can get weaker recovery condition. This shows further that fewer measurements might be needed to recover a block -sparse signal whenever the mixed () minimization methods are used instead of the mixed minimization method. For clarifying this issue more precisely, we can adopt a similar probabilistic method of [17] to derive a simple lower bound on how many random Gaussian measurements are sufficient for (8) to hold with high probability. The result to be verified is in Section 3.
Theorem 3. Let be a matrix with i.i.d. zero-mean unit variance Gaussian entries and for some integer . For any given , if
then the subsequent conclusion is true with probability exceeding : satisfies (8); therefore, any block -sparse signal can be recovered exactly by the solution of the mixed minimization problem (6).
Theorem 3 shows that a block -sparse signal can be recovered exactly with high probability by solving the mixed minimization (6) provided the number of Gaussian measurements satisfies
More detailed remarks on this bound will be presented in Section 3.
The rest of the paper is organized as follows. In Section 2, we present several key lemmas needed for the proofs of our main results. All these lemmas can be regarded as generalizations of the standard non-block-sparse case to the block-sparse case. The proofs of Theorems 2 and 3 are given in Section 3. Numerical experiments are provided in Section 4 to demonstrate the correctness of the theoretical results. We conclude the paper then in Section 5 with some useful remarks.
2. Fundamental Lemmas
In this section, we establish several lemmas necessary for the proofs of the main results.
Lemma 4 (see [8]). Consider
for all , supported on disjoint subsets with , .
In the next lemma, we show that the probability that block restricted isometry constant exceeds a certain scope decays exponentially in certain length of .
Lemma 5 (see [8]). Suppose is an matrix from the Gaussian ensemble; namely, . Let be the smallest value satisfying the block-RIP of over and for some integer . Then, for every , the block restricted isometry constant satisfies the following inequality:
which holds with high probability , where .
Lemma 6. Suppose is an matrix from the Gaussian ensemble; namely, . For any , the following estimation
holds uniformly for with probability exceeding .
Proof. Note that [23] has verified the subsequent conclusion: if is a random matrix of size drawn according to a distribution that satisfies the concentration inequality , where is a constant dependent only on such that for , then, for any set with and any , there holds the estimation
with probability exceeding . By Lemma 5, the concentration inequality is clearly true for Gaussian measurement matrix. Thus Lemma 6 follows.
We also need the following inequality.
Lemma 7 (see [24]). For any fixed and ,
3. Proofs of Theorems
With the preparations made in the last section, we now prove the main results of the paper. Henceforth, we let denote an matrix whose elements are i.i.d. random variables; specifically, . We prove Theorem 2 by using the block-RIP and Lemmas 4 and 7.
Proof of Theorem 2. First notice that assumption (8) implies and further implies the uniqueness of satisfying the observation . Let be a solution of (6). Our goal is then to show that . For this purpose, we let be the signal that is identical to on the index set and to zero elsewhere. Let be the block index set over the nonzero blocks of , and we decompose into a series of vectors such that
Here is the restriction of onto the set and each consists of blocks (except possibly ). Rearrange the block indices such that , for any .
From [9], is the unique sparse solution of (6) being equal to if and only if
for all nonzero signal in the null space of . This is the so-called the null space property (NSP). In order to characterize the NSP more precisely, we consider the following equivalent form: there exists a constant satisfying such that
We proceed by showing that under the assumption of Theorem 2.
In effect, we observe that
For any , if we denote , , , then we have
By Lemma 7, we thus have
that is,
On the other hand, let
It is easy to see that
Since , we have , which means . By the definition of block-RIP, Lemma 4, (22), and (24), it then follows that
By the definition of block-RIP and by using Hölder’s equality, we then get
This together with (24) and (25) implies
Through a straightforward calculation, one can check that the maximum of occurs at and
If , then we clearly have and finish the proof. However, gives that
or, equivalently,
Note that (30) implies and
By simple calculations, we can check that, for any given , inequality (30) is true as long as
that is, condition (8) is satisfied. With this, the proof of Theorem 2 is completed.
Now we adopt the probabilistic methods to analyze how condition (8) can be satisfied; particularly, how many measurements are needed for exact recovery of a block -sparse signal.
Proof of Theorem 3. From Theorem 2, we only need to show that, under condition (9), (8) holds with high probability. In particular, we will proceed to determine how random Gaussian measurements are sufficient for exact recovery of block -sparse signals with a failure probability at most . To this end, we let . Then by Lemma 6, an upper bound for the probability that any submatrix of fails to satisfy the block-RIP (30) is
By inequality , it suffices to show that we can have the following estimation:
So by a simple calculation,
where , . That is, (35) is satisfied as long as (9) is met. This justifies Theorem 3.
Remark 8. From the right-hand side of (35), we can further see that, except the first term related to , other terms in the numerator are only related to and , that is, to the numbers of nonzero blocks and the block size. Obviously, those terms have a smaller contribution to the number of measurements. Basically, Theorem 3 shows that the signal with block-sparsity can be recovered exactly by the solution of the mixed minimization (6) with high probability provided roughly
Obviously, this result generalizes the well-known result in [8] on the Gaussian ensemble for the mixed minimization. We can further see from Theorem 3 that, when specified to , the bound (36) has the same order as Eldar’s result (, see [8]). It is also obvious that is monotonically increasing with respect to , which then implies that decreasing allows fewer necessary measurements for exact recovery of block-sparse signals by the mixed minimization method. Since, when block size , the mixed minimization method degenerates to the standard minimization method, the presented result then provides a theoretical support to such experimental observation reported in the following section.
4. Numerical Experiments
In this section, we conduct two numerical experiments to support the correctness of the obtained theoretical results. Iteratively reweighted least squares (IRLS) method has been proved to be very efficient for the standard minimization. Thus, in the experiments, we adopted IRLS method to solve the following unconstrained smoothed version of (6): where is a regularization parameter and . Note that, through defining a diagonal weighting matrix as , (37) can be transformed to the following weighted least squares minimization problem:
With this, the IRLS algorithm we used can be summarized as follows (more details can be found in [11]).
Step 1. Set the iteration count to zero and . Initialize .
Step 2. Let
and update
Step 3. Terminate the iteration on convergence or when attains a specified maximum number .
Otherwise, set and go to Step 2.
The above algorithm can be seen as a natural generalization of general sparse signal recovery algorithm to the block-sparse setting, which alternates between estimating and redefining the weighting matrix . Though this algorithm is for the unconstrained penalized problem (37), it can still give a good estimation on the minimum of the constrained problem (6) when we choose a sufficient small (such as ). In the following experiments, we set and terminate the algorithm if or .
In our first experiment, we took the signal length , the block size , and the block-sparsity . For independent 100 trails, we first randomly generated the block-sparse signal with values from a Gaussian distribution of mean 0 and standard deviation 1, and then we randomly drew a measurement matrix from Gaussian ensemble. We also took the number of measurements varying from 24 to 124. The purpose of the experiment was then to check the correctness of Theorem 3. We considered four different values of for both the mixed minimization method and the standard minimization method.
The experiment result is shown in Figure 1. It is seen from Figure 1 that, for all the experiment runs, the smaller requires the smaller number of measurements for exact recovery of block sparse signals. This observation is consistent with Theorem 3. On the other hand, for a fixed , the mixed method is clearly superior to the standard method in this block-sparse setting. For instance, the mixed method with only uses 50 Gaussian measurements for the exact recovery, while the number of measurements needed for method is around 90. This observation also partially supports the theoretical assertion of Theorem 3, namely, that incorporating the block structure into recovery method requires fewer measurements for exact recovery.
In our second experiment, we further studied the effect of block size for block-sparse signal recovery. We set and drew a measurement matrix of size from Gaussian ensemble. In this experiment, the block size was changed while keeping the total sparsity fixed. Figure 2 shows the average root mean squares error (RMSE). Here RMSR is defined as in the logarithmic scale over 100 independent random runs. One can easily see from Figure 2 that the recovery performance for the standard method is independent of the active block number , while the recovery errors for the mixed method are significantly better than the standard method when the active block number is far smaller than the total signal sparsity . Since the total sparsity is fixed and larger leads to smaller , as predicted by Theorem 3, the necessary measurement number needed for exact recovery of a block -sparse signal by the mixed method is also reduced. This also demonstrates the advantage of the mixed method over the standard method.
5. Conclusion
In this paper, the number of Gaussian measurements necessary for the exact recovery of a block-sparse signal by the mixed norm minimization has been studied. The main contribution is the derivation of a lower bound on the necessary number of Gaussian measurements for the exact recovery based on a probabilistic analysis with block-RIP. The obtained results are helpful for understanding the recovery capability and algorithm development of the mixed norm minimization approach for sparse recovery and, particularly, facilitate the applications in the development of block-sparse information processing.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work has been supported by the Natural Science Foundation of China under Grant nos. 61303168, 61273020, and 11171272.