Deterministic Sensing Matrices in Compressive Sensing: A Survey
Compressive sensing is a sampling method which provides a new approach to efficient signal compression and recovery by exploiting the fact that a sparse signal can be suitably reconstructed from very few measurements. One of the most concerns in compressive sensing is the construction of the sensing matrices. While random sensing matrices have been widely studied, only a few deterministic sensing matrices have been considered. These matrices are highly desirable on structure which allows fast implementation with reduced storage requirements. In this paper, a survey of deterministic sensing matrices for compressive sensing is presented. We introduce a basic problem in compressive sensing and some disadvantage of the random sensing matrices. Some recent results on construction of the deterministic sensing matrices are discussed.
Consider a scenario that is a vector we wish to recover. Let be a linear measurements of the vector , which is given by where is the measurement matrix or sensing matrix. Because this system is underdetermined, the recovery problem of the vector from the measurement vector is an ill-posed problem. However, two papers by Donoho  and Candès et al.  gave us a breakthrough by exploiting sparsity in recovery problems. The authors show that a sparse signal can be reconstructed from very few measurements by solving via -minimization -minimization or adopting a strategy between and The sufficient conditions for having the solution of to coincide with that of are dependent on either mutual coherence or Restricted Isometry Property (RIP). These conditions are closely related to each other and play an important role in the construction of sensing matrices. Consider is an sensing matrix we investigate. Then its mutual coherence is defined as
Lemma 1 (see ). For an sensing matrix , the Welch bound is given by
The existence and uniqueness of the solution can be guaranteed as soon as the measurement matrix satisfies the RIP of order ; that is, The smallest value of is called the Restricted Isometry Constant (RIC). A strict condition also guarantees exact solution via -minimization. However, the problem remains NP-hard; that is, it cannot be solved in practice. For , there is no numerical scheme to compute solutions with minimal -norm as well. Furthermore, the problem is a convex optimization problem, and in fact, it can be formulated as a linear optimization problem. Then solving via -minimization is efficient with high probability. Hence, most researchers are interested in the recovery via -minimization.
There are two common ways to solve these problems. First, we can exactly recover via -minimization by solving the problem or which is given as The second method is using greedy algorithms for -minimization, such as Matching Pursuit (MP), Orthogonal Matching Pursuit (OMP), or their modifications [4–8].
However, in order to ensure unique and stable reconstruction, the sensing matrix must satisfy some criteria. One of the well-known criteria is -RIP. More attention has been paid to random sensing matrices generated by identical and independent distributions (i.i.d.) such as Gaussian, Bernoulli, and random Fourier ensembles, to name a few. Their applications have been shown in medical images processing , geophysical data analysis , communications [11, 12], and other various signal processing problems. Even though random sensing matrices ensure high probability in reconstruction, they also have many drawbacks such as excessive complexity in reconstruction, significant space requirement for storage, and no efficient algorithm to verify whether a sensing matrix satisfies RIP property with small RIC value. Hence, exploiting specific structures of deterministic sensing matrices is required to solve these problems of the random sensing matrices.
Recently, several deterministic sensing matrices have been proposed. We can classify them into two categories. First are those matrices which are based on coherence [13–15]. Second are those matrices which are based on RIP or some weaker RIPs [16–20]. In this paper, we introduce some highlighted results such as deterministic construction of sensing matrices via algebraic curves over finite fields in term of coherence and chirp sensing matrices, second-order Reed-Muller codes, binary Bose-Chaudhuri-Hocquenghem (BCH) codes, and the sensing matrices with statistical RIP in terms of the RIP.
The rest of this paper is organized as follows. Section 2 introduces some random sensing matrices and their practical disadvantages. In Section 3, we present some highlighted results in terms of deterministic constructions. Section 4 concludes this paper.
2. Random Sensing Matrices and Their Drawbacks
Recall that is the vector we want to recover. Because the number of measurements is much smaller than its dimension , we cannot find a linear identity reconstruction map; that is, unique solution does not exist for all in . However, if we assume that the signal belongs to a certain subset which is the set of all -sparse vectors as for each index set , the -best approximation is given by where can be any norm in . As we noted above, the use of randomly generated sensing matrices has become powerful in compressive sensing. For an upper bound where is a positive constant, the i.i.d. Gaussian matrix achieves the -RIP as well, which guarantees to recover sparse signals with high probability [21, 22]. The condition in (7) is also known to hold for the symmetric Bernoulli distribution case and changed to for the Fourier measurements . For noiseless recovery, it can be stated as follows.
Theorem 2 (see ). If is a -sparse vector and the sensing matrix satisfies then is the unique minimizer to .
In practice, the original signals may be affected by noise, so the recovered signals are not exact, and rather they are almost sparse instead. Hence, some modified criteria were proposed as follows.
Theorem 3 (see ). Suppose that and the noise satisfies . If the sensing matrix has RIP such that then which is the output of the reconstruction algorithm applied to via will obey where the constant depends on sparsity .
A new result on RIC was proposed by Candès as follows.
Theorem 4 (see ). Given and an upper bound of noise , if the sensing matrix has RIP such that then any solution of obeys where and are two positive constants depending on sparsity .
Several inequalities in terms of RIC have been discovered, such as in , in , or in , to name a few. In sum, we can obtain stable and unique solution by using tools from random sensing matrices.
Random matrices are easy to construct and ensure high probability reconstruction. However, they also have many drawbacks. First, storing random matrices requires a lot of storage. Second, there is no efficient algorithm verifying RIP condition. So far, it is not a good approach because of its lack of efficiency. The recovery problems may be difficult when the dimension of the signal becomes large, and we have to construct a measurement matrix that satisfies RIP with a small , such as Theorem 4.
3. Deterministic Sensing Matrices
3.1. Chirp Sensing Matrices
A discrete chirp of length has the form where is the chirp rate and is the base frequency. The full chirp sensing matrix of size can be written as Each matrix is an matrix with columns given by the chirp signals having a fixed chirp rate with base frequency varying from to . For instance, given and , we obtain Hence, we get the deterministic sensing matrix as Note that when , the matrix corresponding to chirp rate becomes the Discrete Fourier Transform (DFT) matrix.
Most of the sensing chirp matrices admit a fast reconstruction algorithm which reduces the complexity to .
3.2. Second-Order Reed-Muller Sensing Matrices
The second-order Reed-Muller code is given as follows: where is a binary symmetric matrix, is a binary vector in , and is the weight of , that is, number of bit-1 entries. In practice, the matrices are set as all-zero matrices or the matrices with zero diagonals. Thus, there are only matrices satisfying this condition, which are , and the functions are real valued. The set forms a basis of . The inner product on is defined as follows. For any two vectors and in , where . The deterministic sensing matrix has the form where is the unitary matrix corresponding to . Note that if we set and , we get an sensing matrix . For instance, let ; then There are only binary symmetric matrices of size satisfying the condition. These are Hence, we get the deterministic sensing matrix as Reconstruction algorithms using the second-order Reed-Muller sensing matrices can outperform the standard compressive sensing using random matrices via -minimization, especially when the original signal is not sparse and the noise is present. Moreover, the nesting of the Delsarter-Goethals sets of the Reed-Muller codes is still feasible if the dimension of the original signal is large [17, 29].
3.3. Binary BCH Matrices
Denote as a divisor of for some integer , and as a primitive th root of unity and assume that is the smallest integer for which divides . If we set , then has order . Define a code over by The BCH code is defined by In other words, if we denote as the null-space of the above matrix of , then the BCH code .
An example of binary matrices formed by BCH code is given as follows. Let , , and , and let be a primitive element of satisfying . Then . The BCH code is the set of 15 tuples that lie in the null space of the matrix Since satisfies , we have as a codeword in the BCH code. The binary matrix is obtained as follows: Since BCH code is cyclic, we can describe it in terms of a generator polynomial which is the smallest degree polynomial having zeros . The advantages of these matrices are deterministic construction, simplicity in sampling process, and reduced computational complexity compared with the DeVoice’s binary sensing matrices. However, the generated matrices formed by BCH codes do not acchieve the RIP constraint yet.
3.4. Sensing Matrices with Statistical Restricted Isometry Property
In , the authors proposed some weaker Statistical Restricted Isometry Properties (StRIPs) defined as follows.
Definition 5 (StRIP). An matrix is called StRIP of order with constant and probability if with respect to a uniform distribution of vector among all -sparse vectors in of the same norm.
Definition 6 (UStRIP). An matrix is called UStRIP of order with constant and probability if it satisfies StRIP and with probability exceeding with respect to a uniform distribution of vector among all -sparse vectors in of the same norm.
These constructions allow recovery methods for which expected performance is sublinear in and quadratic in , compared to the superlinear in of the BP or the MP algorithms. The criteria are simple; however, when satisfied by a deterministic sensing matrix, they guarantee successful recovery in an exponentially small fraction of -sparse signals. The authors also showed that such sensing matrices satisfying these aforementioned properties could be constructed by chirps, second-order Reed-Muller codes, and BCH codes [16–18].
3.5. Deterministic Construction of Sensing Matrices via Algebraic Curves over Finite Fields
In , DeVore used polynomials over finite field to construct binary sensing matrices of size , where is a primer number. Let be a subset of , and let be the set of all polynomials of degree no more than on as There are such polynomials. Any polynomial can be described as a mapping The set of is a subset of . We order the element of as . For given , the set has of order pairs. For each , we denote as the vector indexed on which is defined by where for .
Theorem 7. Let be the matrices with columns , with these columns ordered lexicographically with respect to the coefficients of the polynomial. Then the matrix satisfies the RIP of order with RIC value .
There are several deterministic constructions of sensing matrices via algebraic curves over finite fields called algebraic geometry codes [30–33]. Goppa’s code is one of well-known results which contain many linear codes with many good parameters. Hence, these kinds of sensing matrices are good candidates in reconstruction issues using compressive sensing.
3.6. Binary Sensing Matrices Generated by Unbalanced Expander Graphs
In , a large class of deterministic sensing matrices based on unbalanced expander graphs, that is, the combinatorial structures, was proposed. Denoting , these bipartite graphs are formalized through the following definitions.
Definition 8. A bipartite graph with left vertices, right-vertices, and left-degree is specified by a function , where denotes the th neighbor of . For a set , we write to denote its set of neighbors .
Definition 9. A bipartite graph is a expander if for every set of size , we have .
They constructed a large class of binary and sparse matrices satisfying a different form of the RIP property called RIP- as If the sensing matrix is an adjacency matrix of high-quality unbalanced expander, then the RIP- holds for .
Theorem 10 (see ). Consider any matrix which is the adjacency matrix of an unbalanced expander , , with left degree , such that and . Then the scaled matrix satisfies the RIP with RIC value for some positive constant .
This approach utilizes sparse matrices interpreted as adjacency matrices of sparsity to recover an approximation to the original signal. The new property RIP- suffices to guarantee exact recovery algorithms.
4. Concluding Remarks
In this paper, various deterministic sensing matrices have been investigated and presented in terms of coherence and RIP. The advantages of these matrices, in addition to their deterministic constructions, are the simplicity in sampling and recovery process as well as small storage requirement. It can be possible to make further improvement in both reconstruction efficiency and accuracy using these deterministic matrices in compressive sensing, particularly when some a priori information on location of nonzero components is available.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This research was partly supported by Mid-career Researcher Program through NRF Grant funded by MEST, Korea (No. 2013-030059), and by the MSIP, Korea, in the ICT R&D Program 2013 (KCA-2012-12-911-01-107).
L. R. Welch, “Lower bounds on the maximum cross correlation of signals,” IEEE Transactions on Information Theory, vol. 20, no. 3, pp. 397–399, 1974.View at: Google Scholar
A. Basarab, H. Liebgott, O. Bernard, D. Friboulet, and D. Kouame, “Medical ultrasound image reconstruction using distributed compressive sampling,” in Proceedings of the 10th IEEE International Symposium on Biomedical Imaging (ISBI '13), pp. 628–631, San Francisco, Calif, USA, April 2013.View at: Publisher Site | Google Scholar
S. D. Howard, A. R. Calderbank, and S. J. Searle, “A fast reconstruction algorithm for deterministic compressive sensing using second order reed-muller codes,” in Proceedings of the 42nd Annual Conference on Information Sciences and Systems (CISS '08), pp. 11–15, Princeton, NJ, USA, March 2008.View at: Publisher Site | Google Scholar
R. Berinde, A. C. Gilbert, P. Indyk, H. Karloff, and M. J. Strauss, “Combining geometry and combinatorics: a unified approach to sparse signal recovery,” in Proceedings of the 46th Annual Allerton Conference on Communication, Control, and Computing, pp. 798–805, Urbana-Champaign, Ill, USA, September 2008.View at: Publisher Site | Google Scholar
P. Indyk, “Explicit constructions for compressed sensing of sparse signals,” in Proceedings of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 30–33, San Francisco, Calif, USA, January 2008.View at: Google Scholar
K. R. Davidson and S. J. Szarek, “Local operator theory, random matrices and Banach spaces,” in Handbook of the Geometry of Banach Spaces, W. B. Johnson and J. Lindenstrauss, Eds., vol. 1, pp. 317–366, North-Holland, Amsterdam, The Netherlands, 2001.View at: Google Scholar
S. J. Szarek, “Condition numbers of random matrices,” Journal of Complexity, vol. 7, no. 2, pp. 131–149, 1991.View at: Google Scholar
P. Delsarte and J. M. Goethals, “Alternating bilinear forms over GF(q),” Journal of Combinatorial Theory A, vol. 19, no. 1, pp. 26–50, 1975.View at: Google Scholar
V. D. Goppa, “Codes on algebraic curves,” Doklady Akademii Nauk SSSR, vol. 259, no. 6, pp. 1289–1290, 1981.View at: Google Scholar
M. A. Tsfasman and S. G. Vladut, Algebraic-Geometric Codes, vol. 58 of Mathematics and Its Applications, Springer, Dordrecht, The Netherlands, 1991.