Deterministic Sensing Matrices in Compressive Sensing: A Survey

Abstract

Compressive sensing is a sampling method which provides a new approach to efficient signal compression and recovery by exploiting the fact that a sparse signal can be suitably reconstructed from very few measurements. One of the most concerns in compressive sensing is the construction of the sensing matrices. While random sensing matrices have been widely studied, only a few deterministic sensing matrices have been considered. These matrices are highly desirable on structure which allows fast implementation with reduced storage requirements. In this paper, a survey of deterministic sensing matrices for compressive sensing is presented. We introduce a basic problem in compressive sensing and some disadvantage of the random sensing matrices. Some recent results on construction of the deterministic sensing matrices are discussed.

1. Introduction

Consider a scenario that 𝐱∈ℝ^𝑛 is a vector we wish to recover. Let 𝐲∈ℝ^𝑚(𝑚≪𝑛) be a linear measurements of the vector 𝐱, which is given by 𝐲=𝐀𝐱, (1) where 𝐀 is the measurement matrix or sensing matrix. Because this system is underdetermined, the recovery problem of the vector 𝐱 from the measurement vector 𝐲 is an ill-posed problem. However, two papers by Donoho [1] and Candès et al. [2] gave us a breakthrough by exploiting sparsity in recovery problems. The authors show that a sparse signal can be reconstructed from very few measurements by solving via 𝑙₀-minimization min_{𝐳∈ℝ𝑛}‖𝐳‖₀subjectto𝐀𝐳=𝐲, P₀𝑙₁-minimization min_{𝐳∈ℝ𝑛}‖𝐳‖₁subjectto𝐀𝐳=𝐲, P₁ or adopting a strategy between P₀ and P₁min_{𝐳∈ℝ𝑛}‖𝐳‖_𝑞subjectto𝐀𝐳=𝐲. P_𝑞 The sufficient conditions for having the solution of P₀ to coincide with that of P₁ are dependent on either mutual coherence or Restricted Isometry Property (RIP). These conditions are closely related to each other and play an important role in the construction of sensing matrices. Consider 𝐀=[𝐚₁⋯𝐚_𝑛] is an 𝑚×𝑛 sensing matrix we investigate. Then its mutual coherence is defined as 𝜇(𝐀)𝐚=max^𝑖,𝐚_𝑗𝐚_𝑖2𝐚_𝑗2,𝑖,𝑗=1,…,𝑛. (2)

Lemma 1 (see [3]). For an 𝑚×𝑛 sensing matrix 𝐀, the Welch bound is given by 𝑛−𝑚𝑚()(𝐀)𝑛−1≤𝜇≤1. (3)

The existence and uniqueness of the solution can be guaranteed as soon as the measurement matrix 𝐀 satisfies the RIP of order 𝑘; that is, 1−𝛿_𝑘‖𝐳‖²₂≤‖𝐀𝐳‖²₂≤1+𝛿_𝑘‖𝐳‖²₂,where‖𝐳‖₀≤𝑘. (4) The smallest value of 𝛿_𝑘 is called the Restricted Isometry Constant (RIC). A strict condition 𝛿_2𝑠<1 also guarantees exact solution via 𝑙₀-minimization. However, the problem P₀ remains NP-hard; that is, it cannot be solved in practice. For 0<𝑞≤1, there is no numerical scheme to compute solutions with minimal 𝑙_𝑞-norm as well. Furthermore, the problem P₁ is a convex optimization problem, and in fact, it can be formulated as a linear optimization problem. Then solving via 𝑙₁-minimization is efficient with high probability. Hence, most researchers are interested in the recovery via 𝑙₁-minimization.

There are two common ways to solve these problems. First, we can exactly recover 𝐱 via 𝑙₁-minimization by solving the problem P₁ or P_1,𝜖 which is given as min_{𝐳∈ℝ𝑛}‖𝐳‖₁subjectto‖𝐀𝐳−𝐲‖₂≤𝜖. P_1,𝜖 The second method is using greedy algorithms for 𝑙₀-minimization, such as Matching Pursuit (MP), Orthogonal Matching Pursuit (OMP), or their modifications [4–8].

However, in order to ensure unique and stable reconstruction, the sensing matrix 𝐀 must satisfy some criteria. One of the well-known criteria is 2𝑘-RIP. More attention has been paid to random sensing matrices generated by identical and independent distributions (i.i.d.) such as Gaussian, Bernoulli, and random Fourier ensembles, to name a few. Their applications have been shown in medical images processing [9], geophysical data analysis [10], communications [11, 12], and other various signal processing problems. Even though random sensing matrices ensure high probability in reconstruction, they also have many drawbacks such as excessive complexity in reconstruction, significant space requirement for storage, and no efficient algorithm to verify whether a sensing matrix satisfies RIP property with small RIC value. Hence, exploiting specific structures of deterministic sensing matrices is required to solve these problems of the random sensing matrices.

Recently, several deterministic sensing matrices have been proposed. We can classify them into two categories. First are those matrices which are based on coherence [13–15]. Second are those matrices which are based on RIP or some weaker RIPs [16–20]. In this paper, we introduce some highlighted results such as deterministic construction of sensing matrices via algebraic curves over finite fields in term of coherence and chirp sensing matrices, second-order Reed-Muller codes, binary Bose-Chaudhuri-Hocquenghem (BCH) codes, and the sensing matrices with statistical RIP in terms of the RIP.

The rest of this paper is organized as follows. Section 2 introduces some random sensing matrices and their practical disadvantages. In Section 3, we present some highlighted results in terms of deterministic constructions. Section 4 concludes this paper.

2. Random Sensing Matrices and Their Drawbacks

Recall that 𝐱∈ℝ^𝑛 is the vector we want to recover. Because the number of measurements is much smaller than its dimension (𝑚≪𝑛), we cannot find a linear identity reconstruction map; that is, unique solution does not exist for all 𝐱 in ℝ^𝑛. However, if we assume that the signal 𝐱 belongs to a certain subset Σ_𝑘⊂ℝ^𝑛 which is the set of all 𝑘-sparse vectors as Σ_𝑘(𝐱)=𝐱∈ℝ^𝑛∶𝑥_𝑖=0,𝑖∉𝑇 (5) for each index set 𝑇⊆{1,…,𝑛}, the 𝑘-best approximation is given by 𝜎_𝑘(𝐱)_𝑞=inf‖𝐱−𝐳‖_𝑞,‖𝐳‖₀≤𝑘, (6) where ‖⋅‖_𝑞 can be any norm in 𝐑^𝑛. As we noted above, the use of randomly generated sensing matrices has become powerful in compressive sensing. For an upper bound 𝑘≤𝑐𝑚(),log𝑛/𝑚 (7) where 𝑐 is a positive constant, the i.i.d. Gaussian matrix achieves the 𝑘-RIP as well, which guarantees to recover sparse signals with high probability [21, 22]. The condition in (7) is also known to hold for the symmetric Bernoulli distribution case and changed to 𝑘≤𝑐𝑚/(log𝑛)⁶ for the Fourier measurements [23]. For noiseless recovery, it can be stated as follows.

Theorem 2 (see [24]). If 𝐳∈ℝ^𝑛 is a 𝑘-sparse vector and the sensing matrix 𝐀 satisfies 𝛿_𝑘(𝐀)+𝛿_2𝑘(𝐀)+𝛿_3𝑘(𝐀)<1, (8) then 𝐱 is the unique minimizer to P₁.

In practice, the original signals may be affected by noise, so the recovered signals are not exact, and rather they are almost sparse instead. Hence, some modified criteria were proposed as follows.

Theorem 3 (see [25]). Suppose that 𝐱∈Σ_𝑘 and the noise 𝐞=𝐀𝐳−𝑦 satisfies ‖𝐞‖₂≤𝜖. If the sensing matrix 𝐀 has RIP such that 𝛿_3𝑘(𝐀)+3𝛿_4𝑘(𝐀)<2, (9) then 𝐱^∗ which is the output of the reconstruction algorithm applied to 𝐱 via P_1,𝜖 will obey 𝐱^∗−𝐱₂≤𝑐𝜖, (10) where the constant 𝑐 depends on sparsity 𝑘.

A new result on RIC was proposed by Candès as follows.

Theorem 4 (see [25]). Given 𝐱∈Σ_𝑘(𝐱) and an upper bound of noise ‖𝐞‖₂=‖𝐀𝐳−𝐲‖₂≤𝜖, if the sensing matrix 𝐀 has RIP such that 𝛿_2𝑘<√2−1, (11) then any solution 𝐱^∗ of P_1,𝜖 obeys 𝐱^∗−𝐱₂≤𝑐₁𝜖+𝑐₂𝜎_𝑘(𝐱)₂√𝑘, (12) where 𝑐₁ and 𝑐₂ are two positive constants depending on sparsity 𝑘.

Several inequalities in terms of RIC have been discovered, such as 𝛿_2𝑘<2/(3+√2) in [26], 𝛿_2𝑘<3/(4+√6) in [27], or 𝛿_2𝑘<2/(2+√5) in [28], to name a few. In sum, we can obtain stable and unique solution by using tools from random sensing matrices.

Random matrices are easy to construct and ensure high probability reconstruction. However, they also have many drawbacks. First, storing random matrices requires a lot of storage. Second, there is no efficient algorithm verifying RIP condition. So far, it is not a good approach because of its lack of efficiency. The recovery problems may be difficult when the dimension of the signal becomes large, and we have to construct a measurement matrix that satisfies RIP with a small 𝛿_𝑘, such as Theorem 4.

3. Deterministic Sensing Matrices

3.1. Chirp Sensing Matrices

A discrete chirp of length 𝑚 has the form 𝐴_𝑟,𝜔=1√𝑚exp2𝜋𝑖𝑚𝜔𝑙+2𝜋𝑖𝑚𝑟𝑙²,𝑙,𝜔,𝑟∈ℤ_𝑚, (13) where 𝑟 is the chirp rate and 𝜔 is the base frequency. The full chirp sensing matrix 𝐀 of size 𝑚×𝑚² can be written as 𝐀_chirp=𝐔_𝑟1𝐔_𝑟2⋯𝐔_𝑟𝑘. (14) Each matrix 𝐔_𝑟𝑡(𝑡=1,…,𝜔) is an 𝑚×𝑚 matrix with columns given by the chirp signals having a fixed chirp rate 𝑟_𝑡 with base frequency 𝜔 varying from 0 to 𝑚−1. For instance, given 𝑘=2 and 𝑟,𝑚,𝑙∈{0,1}, we obtain 𝐔_𝑟1=111𝑒^𝑖𝜋,𝐔_𝑟2=𝑒11_𝑖𝜋𝑒^{𝑖𝜋+𝑖2𝜋}. (15) Hence, we get the 2×4 deterministic sensing matrix 𝐀 as 𝐀_chirp=𝐔_𝑟1𝐔_𝑟2=11111𝑒^𝑖𝜋𝑒^𝑖𝜋𝑒^{𝑖𝜋+𝑖2𝜋}. (16) Note that when 𝑟=𝑟₁=0, the matrix 𝐔_𝑟1 corresponding to chirp rate 𝑟₁ becomes the Discrete Fourier Transform (DFT) matrix.

Most of the sensing chirp matrices admit a fast reconstruction algorithm which reduces the complexity to 𝑂(𝑘𝑚log𝑚).

3.2. Second-Order Reed-Muller Sensing Matrices

The second-order Reed-Muller code is given as follows: 𝜙_𝐏,𝐛(𝐚)=(−1)^𝑤(𝐛)√2^𝑝𝑖^{(2𝐛+𝐏𝐚)𝑇𝐚}, (17) where 𝐏 is a 𝑝×𝑝 binary symmetric matrix, 𝐛 is a 𝑝×1 binary vector in ℤ²_𝑝, and 𝑤(𝐛) is the weight of 𝐛, that is, number of bit-1 entries. In practice, the matrices 𝐏 are set as all-zero matrices or the matrices with zero diagonals. Thus, there are only 2^{𝑝(𝑝−1)/2} matrices 𝐏 satisfying this condition, which are {𝐏₁,…,𝐏₂^{𝑝(𝑝−1)/2}}, and the functions {𝜙_𝐏,𝐛(𝐚)} are real valued. The set ℱ_𝐏=𝜙_𝐏,𝐛∣𝐛∈ℤ^𝑝₂ (18) forms a basis of ℤ^𝑝₂. The inner product on ℱ_𝐏 is defined as follows. For any two vectors 𝜙_𝐏,𝐛 and 𝜙_{𝐏′,𝐛′} in ℱ_𝐏, 𝜙_𝐏,𝐛,𝜙_{𝐏′,𝐛′}=1√2^𝑞2^𝑞times,02^𝑝−2^𝑞times, (19) where 𝑞=rank(𝐏−𝐏^′). The deterministic sensing matrix has the form 𝐀_RM=𝐔_𝐏1𝐔_𝐏2⋯𝐔_{𝐏2𝑝(𝑝−1)/22𝑝×2𝑝(𝑝+1)/2}, (20) where 𝐔_𝐏𝐢 is the unitary matrix corresponding to ℱ_𝑃𝑖(𝑖=1,…,2^{𝑝(𝑝−1)/2}). Note that if we set 𝑚=2^𝑝 and 𝑛=2^{𝑝(𝑝+1)/2}, we get an 𝑚×𝑛 sensing matrix 𝐀_RM. For instance, let 𝑝=2; then ℤ²₂=00,01,10,11. (21) There are only 2^2(2−1)/2=2 binary symmetric matrices 𝐏 of size 2×2 satisfying the condition. These are 𝐏₁=0000,𝐏₂=.0110 (22) Hence, we get the deterministic sensing matrix 𝐀 as 𝐀_RM=𝐔_𝐏1𝐔_𝐏2=1−1−111−1−1111−1−111−1−11−11−11−11−11111−1−1−1−1_22×23. (23) Reconstruction algorithms using the second-order Reed-Muller sensing matrices can outperform the standard compressive sensing using random matrices via 𝑙₁-minimization, especially when the original signal is not sparse and the noise is present. Moreover, the nesting of the Delsarter-Goethals sets of the Reed-Muller codes is still feasible if the dimension of the original signal is large [17, 29].

3.3. Binary BCH Matrices

Denote 𝑛 as a divisor of 2^𝑝−1 for some integer 𝑝≥3, and 𝛾∈GF(2^𝑝) as a primitive 𝑛th root of unity and assume that 𝑝 is the smallest integer for which 𝑛 divides 2^𝑝−1. If we set 𝛼=𝛾^{(2𝑝−1)/𝑛}, then 𝛼 has order 𝑛. Define a code 𝒞 over GF(2^𝑝) by 𝐇=1𝛼𝛼²⋯𝛼^(𝑛−1)1𝛼³𝛼⁶⋯𝛼^3(𝑛−1)⋮⋮⋮⋮⋮1𝛼^2𝑖−1𝛼^2(2𝑖−1)⋯𝛼^{(𝑛−1)(2𝑖−1)}⋮⋮⋮⋮⋮1𝛼^2𝑙−1𝛼^2(2𝑙−1)⋯𝛼^{(𝑛−1)(2𝑙−1)}_𝑙×𝑛. (24) The BCH code 𝒞 is defined by 𝐂=𝑐∈𝔽₂^𝑛∣𝐻⋅𝑐^†=0. (25) In other words, if we denote 𝒩 as the null-space of the above matrix of GF(2^𝑝), then the BCH code 𝒞=𝒩∩𝔽₂^𝑛.

An example of binary matrices formed by BCH code is given as follows. Let 𝑛=15, 𝑝=4, and 𝑡=1, and let 𝛾 be a primitive element of GF(2⁴)=GF(16) satisfying 𝛾⁴+𝛾+1=0. Then 𝛼=𝛾^{(2𝑝−1)/𝑛}=𝛾^(24−1)/15=𝛾. The BCH code is the set of 15 tuples that lie in the null space of the matrix 𝐇=1𝛼𝛼²⋯𝛼¹⁴. (26) Since 𝑔(𝑥)=𝑥⁴+𝑥+1𝑥⁴+𝑥³+𝑥²+𝑥+1=𝑥⁸+𝑥⁷+𝑥⁶+𝑥⁴+1 (27) satisfies 𝑔(𝛼³)=0, we have [100010111000000]^𝑇 as a codeword in the BCH code. The binary matrix is obtained as follows: 𝐀_bin=.000100110101111001001101011110010011010111100100010011010111 (28) Since BCH code is cyclic, we can describe it in terms of a generator polynomial which is the smallest degree polynomial having zeros 𝛼,𝛼³,…,𝛼^2𝑖−1,…,𝛼^2𝑙−1. The advantages of these matrices are deterministic construction, simplicity in sampling process, and reduced computational complexity compared with the DeVoice’s binary sensing matrices. However, the generated matrices formed by BCH codes do not acchieve the RIP constraint yet.

3.4. Sensing Matrices with Statistical Restricted Isometry Property

In [18], the authors proposed some weaker Statistical Restricted Isometry Properties (StRIPs) defined as follows.

Definition 5 (StRIP). An 𝑚×𝑛 matrix 𝐴 is called StRIP of order 𝑘 with constant 𝛿 and probability 1−𝜖 if Pr‖𝐀𝐱‖²−‖𝐱‖²≥1−𝜖 (29) with respect to a uniform distribution of vector 𝐱 among all 𝑘-sparse vectors in 𝐑^𝑛 of the same norm.

Definition 6 (UStRIP). An 𝑚×𝑛 matrix 𝐴 is called UStRIP of order 𝑘 with constant 𝛿 and probability 1−𝜖 if it satisfies StRIP and 𝐳∈ℝ^𝑛∶𝐀𝐳=𝐀𝐱={𝐱} (30) with probability exceeding 1−𝜖 with respect to a uniform distribution of vector 𝐱 among all 𝑘-sparse vectors in 𝐑^𝑛 of the same norm.

These constructions allow recovery methods for which expected performance is sublinear in 𝑛 and quadratic in 𝑚, compared to the superlinear in 𝑛 of the BP or the MP algorithms. The criteria are simple; however, when satisfied by a deterministic sensing matrix, they guarantee successful recovery in an exponentially small fraction of 𝑘-sparse signals. The authors also showed that such sensing matrices satisfying these aforementioned properties could be constructed by chirps, second-order Reed-Muller codes, and BCH codes [16–18].

3.5. Deterministic Construction of Sensing Matrices via Algebraic Curves over Finite Fields

In [14], DeVore used polynomials over finite field 𝔽_𝑝 to construct binary sensing matrices of size 𝑝²×𝑝^𝑘+1, where 𝑝 is a primer number. Let {𝑎₀,𝑎₁,…,𝑎_𝑟} be a subset of 𝔽_𝑞, and let 𝑃_𝑟 be the set of all polynomials of degree no more than 𝑟 on 𝔽_𝑝 as 𝑃_𝑟=𝑓∈𝔽_𝑝[𝐱]∶deg𝑓≤𝑘. (31) There are 𝑛=𝑝^𝑟+1 such polynomials. Any polynomial 𝑃∈𝑃_𝑟 can be described as a mapping 𝑃∶𝔽_𝑝⟶𝐅_𝑝,𝑥⁰,𝑥¹,…,𝑥^𝑟⟼𝑃(𝐱)=𝑎₀𝑥⁰+𝑎₁𝑥¹+⋯+𝑎_𝑟𝑘𝑥^𝑟. (32) The set of (𝐱,𝑃(𝐱)) is a subset of 𝔽_𝑝×𝔽_𝑝. We order the element of 𝔽_𝑝×𝔽_𝑝 as (0,0),(0,1),…,(𝑝−1,𝑝−1). For given 𝔽_𝑝, the set 𝔽_𝑝×𝔽_𝑝 has 𝑚=𝑝² of order pairs. For each 𝑃∈𝑃_𝑟, we denote 𝐯_𝑃 as the vector indexed on 𝔽_𝑝×𝔽_𝑝 which is defined by 𝑃_0,0,…,𝑃_0,𝑝−1,𝑃_1,0,…,𝑃_1,𝑝−1,…,𝑃_𝑝−1,0,…,𝑃_{𝑝−1,𝑝−1}^𝑇, (33) where 𝑃_𝑖,𝑗=1,if𝑃(𝑖)=𝑗,0,otherwise (34) for 𝑖,𝑗=0,1,…,𝑝−1.

Theorem 7. Let 𝐀₀ be the 𝑚×𝑛 matrices with columns 𝐯_𝑃, 𝑃∈𝑃_𝑟 with these columns ordered lexicographically with respect to the coefficients of the polynomial. Then the matrix 𝐀=(1/√𝑝)𝐀₀ satisfies the RIP of order 𝑘<𝑝/𝑟+1 with RIC value 𝛿_𝑘=(𝑘−1)𝑟/𝑝.

There are several deterministic constructions of sensing matrices via algebraic curves over finite fields called algebraic geometry codes [30–33]. Goppa’s code is one of well-known results which contain many linear codes with many good parameters. Hence, these kinds of sensing matrices are good candidates in reconstruction issues using compressive sensing.

3.6. Binary Sensing Matrices Generated by Unbalanced Expander Graphs

In [20], a large class of deterministic sensing matrices based on unbalanced expander graphs, that is, the combinatorial structures, was proposed. Denoting [𝑛]={1⋯𝑛}, these bipartite graphs are formalized through the following definitions.

Definition 8. A bipartite graph with 𝑛 left vertices, 𝑚 right-vertices, and left-degree 𝑑 is specified by a function Γ∶[𝑛]×[𝑑]→[𝑚], where Γ(𝑥,𝑦) denotes the 𝑦th neighbor of 𝑥. For a set 𝑆⊂[𝑁], we write Γ(𝑆) to denote its set of neighbors {Γ(𝑥,𝑦)∶𝑥∈𝑆,𝑦∈[𝐷]}.

Definition 9. A bipartite graph Γ∶[𝑛]×[𝑑]→[𝑚] is a (𝐾,𝐴) expander if for every set 𝑆⊂[𝑛] of size 𝑘, we have |Γ(𝑆)|≥𝐴⋅𝐾.

They constructed a large class of binary and sparse matrices satisfying a different form of the RIP property called RIP-𝑝 as 1−𝛿_𝑘‖𝐳‖^𝑝_𝑝≤‖𝐀𝐳‖^𝑝_𝑝≤1+𝛿_𝑘‖𝐳‖^𝑝_𝑝,where‖𝐳‖₀≤𝑘. (35) If the sensing matrix 𝐀 is an adjacency matrix of high-quality unbalanced expander, then the RIP-𝑝 holds for 1≤𝑝≤11+𝑂(1)/log(𝑛).

Theorem 10 (see [19]). Consider any 𝑚×𝑛 matrix 𝐀₀ which is the adjacency matrix of an (𝑘,𝜖) unbalanced expander 𝐺=(𝐴,𝐵,𝐸), |𝐴|=𝑛, |𝐵|=𝑚 with left degree 𝑑, such that 1/𝜖 and 𝑑<𝑛. Then the scaled matrix 𝐀=(1/√𝑑)𝐀₀ satisfies the RIP with RIC value 𝛿=𝑐𝜖 for some positive constant 𝑐>1.

This approach utilizes sparse matrices interpreted as adjacency matrices of sparsity to recover an approximation to the original signal. The new property RIP-𝑞 suffices to guarantee exact recovery algorithms.

4. Concluding Remarks

In this paper, various deterministic sensing matrices have been investigated and presented in terms of coherence and RIP. The advantages of these matrices, in addition to their deterministic constructions, are the simplicity in sampling and recovery process as well as small storage requirement. It can be possible to make further improvement in both reconstruction efficiency and accuracy using these deterministic matrices in compressive sensing, particularly when some a priori information on location of nonzero components is available.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research was partly supported by Mid-career Researcher Program through NRF Grant funded by MEST, Korea (No. 2013-030059), and by the MSIP, Korea, in the ICT R&D Program 2013 (KCA-2012-12-911-01-107).