A Sharp RIP Condition for Orthogonal Matching Pursuit

Abstract

A restricted isometry property (RIP) condition 𝛿_𝐾+√𝐾𝜃_𝐾,1<1 is known to be sufficient for orthogonal matching pursuit (OMP) to exactly recover every K-sparse signal x from measurements 𝑦=Φ𝑥. This paper is devoted to demonstrate that this condition is sharp. We construct a specific matrix with 𝛿_𝐾+√𝐾𝜃_𝐾,1=1 such that OMP cannot exactly recover some K-sparse signals.

1. Introduction

Compressive sampling (compressed sensing, CS) is known as a new type of sampling theory that one can reconstruct a high dimensional spare signal from a small number of linear measurements at the sub-Nyquist rate [1–3]. Nowadays, the CS technique has attracted considerable attention from across a wide array of fields like applied mathematics, statistics, and engineering, including signal processing areas such as MR imaging, speech processing, and analog to digital conversion. The basic problem in CS is to reconstruct the unknown sparse signal 𝑥 from measurements: 𝑦=Φ𝑥, (1) where Φ is an 𝑀×𝑁(𝑀≪𝑁) sampling matrix. Suppose Φ=(Φ₁,Φ₂,…,Φ_𝑁), where Φ_𝑖 denotes the 𝑖th column of Φ. Throughout the paper, we will assume that the columns of Φ are normalized; that is, ‖Φ_𝑖‖₂=1 for 𝑖=1,2,…,𝑁.

It is well understood that under some assumptions on the sampling matrix Φ, the unknown sparse signal 𝑥 can be reconstructed by solving the 𝑙₀-minimization problem: min‖𝑥‖₀subjectto𝑦=Φ𝑥, (2) where ‖𝑥‖₀ denotes the number of nonzero entries of 𝑥. We say a signal 𝑥 is 𝐾-sparse when ‖𝑥‖₀≤𝐾.

However, the optimization problem is NP-hard, so one seeks computationally efficient algorithms to approximate the sparse signal 𝑥, such as greedy algorithm, 𝑙₁ minimization, and 𝑙_𝑝(0<𝑝<1) minimization [4–6].

Orthogonal matching pursuit (OMP), which is a canonical greedy algorithm, has receive much attention in solving the problem (2), due to its ease of implementation and low complexity. Algorithm 1 can be described below. Until recently, many popular generalizations of OMP are introduced, for example, OMMP and KOMP; for details, see [7, 8].

Input: Sampling matrix Φ, observation 𝑦 Output: Reconstructed sparse vector 𝑥∗ and index set INITIALIZATION: Let the index set Ω0=Ø and the residual 𝑟0=𝑦. Let the iteration counter 𝑡=1. IDENTIFICATION: Choose the index 𝑖 subject to |Φ𝑇𝑖𝑟𝑡−1|>max𝑗≠𝑖|Φ𝑇𝑗𝑟𝑡−1|. UPDATE: Add the new index 𝑖 to the index set: Ω𝑡=Ω𝑡−1∪𝑖, and update the signal and the residual              𝑥𝑡|Ω𝑡=argmin𝑧||𝑦−ΦΩ𝑡𝑧||2,𝑥𝑡|Ω𝑡=0;                𝑟𝑡=𝑦−Φ𝑥𝑡. If 𝑟𝑡=0, stop the algorithm. Otherwise, update the iteration counter 𝑡=𝑡+1 and return to Step IDENTIFICATION.

Algorithm 1 

Orthogonal matching pursuit—OMP (Φ,𝑦).

The mutual incoherence property (MIP) introduced in [9] is an important tool to analyze the performance of OMP. The MIP requires the mutual coherence 𝜇 of the sampling matrix Φ to be small, where 𝜇 is defined as 𝜇=max_𝑖≠𝑗Φ^𝑇_𝑖Φ_𝑗. (3) Tropp has shown that the MIP condition (2𝐾−1)𝜇<1 is sufficient for OMP to exactly recover every 𝐾-sparse signal [6]. This condition is proved to be sharp in [10].

The restricted isometry property (RIP) is also widely used in studying a large number of algorithms for sparse recovery in CS, which is introduced in [11]. A matrix Φ satisfies the RIP of order 𝐾 with the restricted isometry constant (RIC) 𝛿_𝐾 if 𝛿_𝐾 is the smallest constant such that 1−𝛿_𝐾‖𝑥‖²₂≤‖Φ𝑥‖²₂≤1+𝛿_𝐾‖𝑥‖²₂ (4) holds for all 𝐾-sparse signal 𝑥. A related quantity of the restricted orthogonality constant (ROC) 𝜃_𝐾,𝐾′ is defined as the smallest quantity such that Φ𝑥,Φ𝑥^′≤𝜃_𝐾,𝐾′‖𝑥‖₂⋅𝑥^′₂ (5) holds for all disjoint support 𝐾-sparse signal 𝑥 and 𝐾^′-sparse signal 𝑥^′. It is first shown by Davenport and Wakin that the RIP condition 𝛿_𝐾+1<13√𝐾 (6) can guarantee that OMP will exactly recover every 𝐾-sparse signal [12]. The sufficient condition is then improved to 𝛿_𝐾+1<1/(1+2√𝐾) [13], 𝛿_𝐾+1<1/√2𝐾 [14], 𝛿_𝐾+1<1/(1+√𝐾) [7], and 𝛿_𝐾+√𝐾𝜃_𝐾,1<1 [15, 16]. By contrast, Mo and Shen have given a counterexample, a matrix with 𝛿_𝐾+1=1/√𝐾 where OMP fails for some 𝐾-sparse signals [17]. The main result of this note is to show that the sufficient RIP condition 𝛿_𝐾+√𝐾𝜃_𝐾,1<1 (7) is sharp for OMP.

2. Main Result

Theorem 1. For any given positive integer 𝐾≥1, there exist a 𝐾-sparse signal 𝑥 and a matrix Φ with the restricted isometry constant 𝛿_𝐾+√𝐾𝜃_𝐾,1=1 (8) for which OMP fails in 𝐾 iterations.

Proof. For any given positive integer 𝐾≥1, let Φ=Φ_{𝑖𝑗(2𝐾−1)×(2𝐾−1)}, (9) where Φ_𝑖𝑗=0𝑖<𝑗,2𝐾⋅−𝑖2𝐾−1𝑖(),𝑖+1𝑖=𝑗2𝐾⋅12𝐾−1𝑖().𝑖+1𝑖>𝑗 (10)
By simple calculation, we can get Φ_𝑗²₂=2𝐾⋅𝑗2𝐾−1𝑗²+𝑗+1^2𝐾−1_𝑖=𝑗+11𝑖()=𝑖+12𝐾⋅𝑗2𝐾−1+1𝑗+1−1𝑗+11𝑗+2+⋯+−12𝐾−1Φ2𝐾=1,_𝑙,Φ_𝑗=2𝐾⋅−𝑗2𝐾−1𝑗+𝑗+1^2𝐾−1_𝑖=𝑗+11𝑖()=𝑖+12𝐾⋅−12𝐾−1+1𝑗+1−1𝑗+11𝑗+2+⋯+−12𝐾−112𝐾=−2𝐾−1 (11) for any integers 1≤𝑙<𝑗≤2𝐾−1.
Thus, for any index set Λ whose cardinality is 𝐾, we have Φ^𝑇_ΛΦ_Λ=11−12𝐾−1⋯−⋱−12𝐾−112𝐾−1⋯−12𝐾−1. (12) It is obvious that the eigenvalues {𝜆_𝑖}^𝐾_𝑖=1 of Φ^𝑇_ΛΦ_Λ are 𝜆₁=⋯=𝜆_𝐾−11=1+,𝜆2𝐾−1_𝐾=1−𝐾−1.2𝐾−1 (13) Therefore, the restricted isometry constant 𝛿_𝐾 of Φ is (𝐾−1)/(2𝐾−1).
Now, we turn to calculate the restricted orthogonality constant 𝜃_𝐾,1. In view of (11), we may, without loss of generality, assume that 𝑥=(𝑥₁,…,𝑥_𝐾,0,…,0)^𝑇 and 𝑥^′=(0,…,0,𝑥^′_𝐾+10,…,0)^𝑇. We have 𝜃_𝐾,1=maxΦ𝑥,Φ𝑥^′‖𝑥‖₂⋅𝑥^′₂=maxΦ𝑥,Φ_𝐾+1‖𝑥‖₂1=max⋅∑2𝐾−1^𝐾𝑖=1𝑥_𝑖‖𝑥‖₂=√𝐾.2𝐾−1 (14) The last equality holds when 𝑥₁=⋯=𝑥_𝐾. It is easy to check that 𝛿_𝐾+√𝐾𝜃_𝐾,1=𝐾−1+√2𝐾−1√𝐾⋅𝐾2𝐾−1=1. (15)
Let 𝑥=(1,…,1_𝐾,0,…,0)^𝑇∈𝐑^2𝐾−1; we have 𝑆_𝑗=Φ𝑥,Φ_𝑗=𝐾{}.2𝐾−1,∀𝑗∈1,2,…,2𝐾−1 (16) This implies that OMP fails in the first iteration. The proof is complete.

3. Discussion

In this paper, we showed that the RIP condition 𝛿_𝐾+√𝐾𝜃_𝐾,1<1 is sharp for orthogonal matching pursuit to exactly recover every 𝐾-sparse signal 𝑥 from measurements 𝑦=Φ𝑥. It is worth discussing the relations between our sharp RIP condition and that in two relative papers [10, 17]. First of all, it follows from the facts that 𝛿_𝐾<(𝐾−1)𝜇 and 𝜃_𝐾,1<√𝐾𝜇 that the sharp RIP condition 𝛿_𝐾+√𝐾𝜃_𝐾,1<1 in this paper is weaker than the sharp MIP condition (2𝐾−1)𝜇<1 in [10]. Moreover, our result is also stronger than the previous RIP condition. The condition 𝛿_𝐾+√𝐾𝜃_𝐾,1<1 in this paper is necessary and sufficient for OMP, while the previous necessary RIP condition 𝛿_𝐾<1/√𝐾 in [17] is not sufficient. Therefore, the result in the paper may guide the practitioners to apply OMP properly in sparse recovery.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant nos. 11271060, U0935004, U1135003, 11071031, and 11290143, 11101096), the Guangdong Natural Science Foundation (Grant no. S2012010010376), and the Guangdong University and Colleges Technology Innovation Projects (Grant no. 2012KJCX0048).