Recovery of High-Dimensional Sparse Signals via ℓ₁-Minimization

Abstract

We consider the recovery of high-dimensional sparse signals via ℓ₁-minimization under mutual incoherence condition, which is shown to be sufficient for sparse signals recovery in the noiseless and noise cases. We study both ℓ₁-minimization under the ℓ₂ constraint and the Dantzig selector. Using the two ℓ₁-minimization methods and a technical inequality, some results are obtained. They improve the results of the error bounds in the literature and are extended to the general case of reconstructing an arbitrary signal.

1. Introduction

The problem of recovering a high-dimensional sparse signal based on a small number of measurements, possibly corrupted by noise, has attracted much recent attention. In the existing literature on sparse signals recovery and compressed sensing (see [1–13] and references therein), the emphasis is on assessing sparse signal 𝑤∈𝑅^𝑛 from an observation𝑦∈𝑅^𝑚: 𝑦=𝐴𝑤+𝑧, (1) where the matrix 𝐴∈𝑅^𝑚×𝑛 with 𝑚≪𝑛 is given and 𝑧∈𝑅^𝑚 is a vector of measurement errors. The goal is to reconstruct the unknown vector 𝑤∈𝑅^𝑛 based on 𝑦 and 𝐴. Throughout the paper, we will assume that the columns of 𝐴 are standardized to have unit ℓ₂-norm.

When noise is present, there are two well-known ℓ₁-minimization methods, which are well suited for recovering sparse signals. One is ℓ₁-minimization under the ℓ₂-constraint on the residuals: 𝑃₁𝑤=argmin_𝑤‖𝑤‖₁∶𝐴𝑤−𝑦₂≤𝜀. (2)

Another method, called Dantzig selector, is proposed by Candes and Tao [4]. The Dantzig selector solves the sparse recovery problem through ℓ₁-minimization with a constraint on the correlation between the residuals and the column vectors of 𝐴: 𝑃₂𝑤=argmin_𝑤‖𝑤‖₁∶𝐴^𝑇𝐴𝑤−𝑦_∞≤𝜀. (3)

It is clear that regularity conditions are needed in order for these problems to be well behaved. Over the last few years, many interesting results for recovering sparse signals have been obtained in the framework of the mutual incoherence property (MIP) introduced by Donoho and Huo [14]. The MIP requires the pairwise correlations among the column vectors of 𝐴 to be small. Let 𝜇=max_𝑖≠𝑗𝐴_𝑖,𝐴_𝑗. (4) See, for example, [8, 10, 14, 15].

It was first shown by Donoho and Huo [14], in the noiseless case for the setting where 𝐴 is a concatenation of two square orthogonal matrices, that 1𝑘<21𝜇+1, (5) ensuring that the exact recovery of 𝑤 when 𝑤 has at most 𝑘 nonzero entries (such a signal is called 𝑘-sparse). This result was then extended in the noiseless case in [11, 16] to a general dictionary 𝐴.

Stronger MIP conditions have been used in the literature to guarantee stable recovery of sparse signals in the noisy case. When noise is assumed to be bounded in ℓ₂-norm, Donoho et al. [15] showed that sparse signals can be recovered approximately through ℓ₁-minimization, with the error being at worst proportional to the input noise level, when 1𝑘<41𝜇+1. (6)

The results in [17] imply that 2𝑘<3+√61𝜇1+1≈0.368𝜇+1 (7) is sufficient for stable recovery. And Tseng [18] used 1𝑘<2𝜇1−𝑂𝜇+1. (8)

Cai et al. [19] have showed that the condition 1𝑘<21𝜇+1 (9) is not only sufficient but in fact sharp for stable recovery with bounded noise as well as Gaussian noise.

In this paper, we consider the problem of recovering a high-dimensional sparse signal via two well ℓ₁-minimization methods under the condition 𝑘<(1/2)(1/𝜇+1). We study both ℓ₁-minimization under the ℓ₂ constraint (𝑃₁) and the Dantzig selector (𝑃₂). Using the two methods and a technical inequality, we give some results, which slightly improve those in [19]. Moreover, we obtain some results when unknown vector is not 𝑘-sparse in the noise case and noiseless case.

The rest of the paper is organized as follows. In Section 2, some basic notation and definitions are reviewed; an elementary inequality, which allow us to make finer analysis of the sparse recovery problem, is introduced. We begin the analysis of ℓ₁-minimization methods for sparse signals recovery by considering the exact recovery in the noise case in Section 3; our results are similar to those in [19] and to some extent, we provide tighter error bounds than the existing results in the literature. In Section 4, we consider the case of unknown vector which is not 𝑘-sparse under the condition 𝑘<(1/2)(1/𝜇+1). We give some facts and the proofs of the theorems in Section 5.

2. Preliminaries

We begin by introducing basic notation and definitions and then develop an important inequality which will be used in proving our main results.

For a vector 𝑣=(𝑣_𝑖)∈ℝ^𝑛, we will denote by supp(𝑣)={𝑖∶𝑣_𝑖≠0} the support of a vector 𝑣. We use the standard notation ‖𝑣‖_𝑞=(∑^𝑝_𝑖=1|𝑣_𝑖|^𝑞)^1/𝑞 to denote the ℓ_𝑞-norm of the vector of 𝑣. Moreover, a vector 𝑣 is said to be 𝑘-sparse if |supp(𝑣)|≤𝑘. We also treat a vector 𝑣=(𝑣_𝑖)∈ℝ^𝑛 as a function 𝑣∶{1,2,…,𝑛}→ℝ by assigning 𝑣(𝑖)=𝑣_𝑖.

We now introduce a useful elementary inequality, which is used in the proofs of the theorems.

Proposition 1. Let 𝑘<𝑛 be positive integers. For any descending chain of real numbers 𝑥₁≥𝑥₂≥⋯≥𝑥_𝑘≥𝑥_𝑘+1≥⋯≥𝑥_𝑛−1≥𝑥_𝑛≥0, (10) One has 𝑥²_𝑘+1+𝑥²_𝑘+2+⋯+𝑥²_𝑛≤𝑥_𝑘+1+𝑥_𝑘+2+⋯+𝑥_𝑛𝑥₁+𝑥₂+⋯+𝑥_𝑘𝑘 (11)

Proof. Since 𝑥_𝑖≥𝑥_𝑗 for 𝑖<𝑗, we have 𝑥²_𝑘+1+𝑥²_𝑘+2+⋯+𝑥²_𝑛≤𝑥_𝑘𝑥_𝑘+1+𝑥_𝑘+1𝑥_𝑘+2+⋯+𝑥_𝑛−1𝑥_𝑛=𝑘𝑥_𝑘𝑥_𝑘+1+𝑥_𝑘+1𝑥_𝑘+2+⋯+𝑥_𝑛−1𝑥_𝑛𝑘≤𝑥_𝑘+1+𝑥_𝑘+2+⋯+𝑥_𝑛𝑥₁+𝑥₂+⋯+𝑥_𝑘𝑘. (12)

3. Recovery of 𝑘-Sparse Signals

As previously above, the condition (5) has been proved to guarantee the recovery of 𝑘-sparse signal in noiseless case. Cai et al. [19] have shown that this condition is also sufficient for stable reconstruction of 𝑘-sparse signals in the noisy case when the error is in a bounded set. We will also give the results for reconstruction of 𝑘-sparse signals both in the noiseless and noisy cases with error bounded, which are proved using different methods from [19].

Theorem 2. Consider the model (1) with ‖𝑧‖₂≤𝜀. Suppose that 𝑤 is k-sparse and 𝑤 is the solution of ℓ₁-minimization problem (𝑃₁). Then, under the condition 𝑘<(1/2)(1/𝜇+1), ‖𝑤−𝑤‖₂≤221+(𝑘−1)𝜇1−(2𝑘−1)𝜇𝜀. (13)

We now consider sparse recovery of 𝑘-sparse signals with error in a different bounded set. Candes and Tao [4] treated the sparse signal recovery in the Gaussian noise case by solving minimization min‖𝑤‖₁subjectto𝐴^𝑇𝑦−𝐴𝑤_∞≤𝜂, (14) with bounded set {𝑧∶‖𝐴^𝑇𝑧‖_∞≤𝜂} and referred the solution as the Dantzig Selector. The following result shows that the condition 𝑘<(1/2)(1/𝜇+1) is also sufficient when the error is in the bounded set {𝑧∶‖𝐴^𝑇𝑧‖_∞≤𝜀}.

Theorem 3. Consider the model (1) with ‖𝐴^𝑇𝑧‖_∞≤𝜀. Suppose that 𝑤 is k-sparse and 𝑤 is the solution of ℓ₁-minimization problem (𝑃₂). Then, under the condition 𝑘<(1/2)(1/𝜇+1), ‖𝑤−𝑤‖₂≤2√2𝑘1−(2𝑘−1)𝜇𝜀. (15)

Remark 4. We consider the stable recovery of sparse signals with error in the ℓ₂-ball; for example, 𝐵⊂ℝ^𝑛 is a bounded set. 𝐵 is taken to be {0} in the noiseless case and can be 𝐵^ℓ2={𝑧∶‖𝑧‖₂≤𝜂} or 𝐵^𝐷𝑆={𝑧∶‖𝐴^𝑇𝑧‖_∞≤𝜂} in the noisy case.
Note that 2𝑘𝜇<𝜇+1, under the condition 𝑘<(1/2)(1/𝜇+1), the result of Theorem 2 is equivalent to ‖𝑤−𝑤‖₂≤3−𝜇()𝜇,1−2𝑘−1𝜀+𝜂 (16) and Theorem 3 is equivalent to ‖𝑤−𝑤‖₂≤√2𝑘()𝜇.1−2𝑘−1𝜀+𝜂 (17)

To some extent, Theorem 2 improves Theorem  2.1 in [19], while Theorem 3 is improved using different method from [19] and gets the same results as Theorem  2.2 in [19].

4. Recovery of Approximately 𝑘-Sparse Signals

In the previous section, the focus was on recovering 𝑘-sparse signals. As discussed in [17, 19, 20], our results can also be stated in the general setting of reconstructing an arbitrary signal under the condition 𝑘<(1/2)(1/𝜇+1).

We begin in this section by considering the problem of exact recovery of spares signals when no noise is present. This is an interesting problem in itself and has been considered in a number of papers; see, for example, [9, 11, 17, 21]. More importantly, the solutions to this “clean” problem shed light on the noisy case.

When 𝑤 is not 𝑘-sparse, ℓ₁-minimization can also recover 𝑤 with accuracy if 𝑤 has good 𝑘-term approximation. For a general vector 𝑤∈𝑅^𝑛, denote by 𝑤_max(𝑘) with all but the 𝑘-largest entries (in absolute value) set to zero and 𝑤_−max(𝑘) the vector 𝑤 with the 𝑘-largest entries (in absolute value) set to zero.

Theorem 5. Let 𝐴∈𝑅^𝑚×𝑛. Suppose that 𝑘≥1 satisfies 𝑘<(1/2)(1/𝜇+1) and 𝑤 is the solution of the following ℓ₁-minimization problem (𝑃)min‖𝑤‖₁𝑠𝑢𝑏𝑗𝑒𝑐𝑡𝑡𝑜𝑦=𝐴𝑤. (18) Then ‖𝑤−𝑤‖₂≤2𝜇√2𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (19)

We now turn to the noisy case. Suppose that 𝐴∈𝑅^𝑚×𝑛 and 𝑦=𝐴𝑤+𝑧, where the noise 𝑧 is bounded. We will specifically consider two cases: ‖𝑧‖₂≤𝜀 and ‖𝐴^𝑇𝑧‖_∞≤𝜀. We will first consider the case ‖𝑧‖₂≤𝜀.

Theorem 6. Consider the model (1) with 𝑧 satisfying ‖𝑧‖₂≤𝜀. Suppose that 𝑘≥1 satisfies 𝑘<(1/2)(1/𝜇+1) and 𝑤 is the solution of the ℓ₁-minimization problem (𝑃₁). Then ‖𝑤−𝑤‖₂≤2𝜀21+(𝑘−1)𝜇+2𝜇√2𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (20)

We next turn to the case ‖𝐴^𝑇𝑧‖_∞≤𝜀, which is called Dantzig selector.

Theorem 7. Consider the model (1) with ‖𝐴^𝑇𝑧‖_∞≤𝜀. Suppose that 𝑘≥1 satisfies 𝑘<(1/2)(1/𝜇+1) and 𝑤 is the solution of ℓ₁-minimization problem (𝑃₂). Then ‖𝑤−𝑤‖₂≤2𝜀√2𝑘+2𝜇√2𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (21)

We have so far focused on stable recovery with bounded error. The results can be extended directly to the Gaussian noise case. This is due to the fact that Gaussian noise is “essentially bounded.” See, for example, [17, 19, 20].

5. The Proofs of the Theorems

Before giving the proofs of the theorems, we introduce three widely used facts, which are useful for the proofs.(A) The following fact is well known in the recovery of sparse signals. Let 𝑥∈ℝ^𝑛 be any 𝑘-sparse signal and 𝐴∈𝑅^𝑚×𝑛; then 1−(𝑘−1)𝜇‖𝑥‖²₂≤‖𝐴𝑥‖²₂≤1+(𝑘−1)𝜇‖𝑥‖²₂, (22)where 𝜇 is defined by (4); see, for example, [17, 18, 22, 23].

Let 𝑤 be a solution to the minimization problem; then by definition ‖𝑤‖₁≤‖𝑤‖₁. Let 𝛿=𝑤−𝑤, and 𝛿_𝑆0=𝛿𝐼supp(𝑤). Here, 𝐼_𝐽 denotes the indicator function of a set 𝐽; that is, 𝐼_𝐽(𝑗)=1 if 𝑗∈𝐽 and 0 if 𝑗∉𝐽.(B) The following is a widely used fact (see, e.g., [4, 7, 14, 17]): 𝛿−𝛿_𝑆01≤𝛿_𝑆01. (23)

This follows directly from the fact that ‖𝑤‖₁≥‖𝑤‖₁=‖𝑤+𝛿‖₁=𝑤+𝛿_𝑆01+𝛿−𝛿_𝑆01≥‖𝑤‖₁−𝛿_𝑆01+𝛿−𝛿_𝑆01. (24)(C) The following fact, which is based on the minimality of 𝑤, has been widely used; see for example, [14, 19, 20]: 𝛿_𝑇01≥𝛿_𝑇^𝑐₀₁, (25)where 𝑇₀ is the support of 𝑤. This follows directly from the fact that ‖𝑤‖₁≥‖𝑤‖₁=‖𝑤+𝛿‖₁=𝑤+𝛿_𝑇01+𝛿^𝑐_𝑇01≥‖𝑤‖₁−𝛿_𝑇01+𝛿_𝑇^𝑐₀₁. (26)

Proof of Theorem 2. The proof makes use of the ideas from [17, 19, 22].
Let 𝛿=𝑤−𝑤. Rearranging the indices if necessary, we assume that 𝛿₍₁₎≥𝛿₍₂₎≥⋯≥𝛿_(𝑘+1)≥𝛿_(𝑘+2)≥⋯. (27)
Let 𝑇₁={1,2,…,𝑘} and 𝑇₂ be the support of 𝑤; then, from the fact (C), 𝛿_𝑇21≥𝛿_𝑇^𝑐₂₁. (28)
Note that 𝑇^𝑐₂∩𝑇₁ and 𝑇₂∩𝑇^𝑐₁ both have 𝑘−|𝑇₁∩𝑇₂| elements, so we have 𝛿_𝑇^𝑐_2∩𝑇11≥𝛿_𝑇2∩𝑇^𝑐₁₁. (29)
We will show that this implies that 𝛿_𝑇11≥𝛿_𝑇^𝑐₁₁. (30)
In fact 𝛿_𝑇11=𝛿_{𝑇1∩𝑇21}+𝛿_𝑇1∩𝑇^𝑐₂₁=𝛿_𝑇21−𝛿_𝑇2∩𝑇^𝑐₁₁+𝛿_𝑇1∩𝑇^𝑐₂₁≥𝛿_𝑇^𝑐₂₁−𝛿_𝑇2∩𝑇^𝑐₁₁+𝛿_𝑇1∩𝑇^𝑐₂₁=𝛿_𝑇^𝑐_1∩𝑇^𝑐₂₁−𝛿_𝑇2∩𝑇^𝑐₁₁+2𝛿_𝑇1∩𝑇^𝑐₂₁=𝛿_𝑇^𝑐₁₁−2𝛿_𝑇2∩𝑇^𝑐₁₁+2𝛿_𝑇1∩𝑇^𝑐₂₁≥𝛿_𝑇^𝑐₁₁. (31)
For simplicity, partition {1,2,…,𝑛} into the following sets: 𝑆₀={1,2,…,𝑘},𝑆₁=𝑘+1,𝑘+2,…,𝑘+𝑘₁,𝑆₂=𝑘+𝑘₁+1,𝑘+𝑘₁+2,…,𝑘+𝑘₁+𝑘₂,…, (32) where 𝑘_𝑗(𝑗>0) is an positive integer.
Note that 𝑘<(1/2)(1/𝜇+1) is equivalent to (2𝑘−1)𝜇<1 or 1−(2𝑘−1)𝜇>0. Now 𝐴𝛿,𝐴𝛿_𝑆0=𝐴𝛿_𝑆0,𝐴𝛿_𝑆0+𝐴𝛿−𝛿_𝑆0,𝐴𝛿_𝑆0≥1−(𝑘−1)𝜇𝛿_𝑆0²₂−𝜇𝛿−𝛿_𝑆01𝛿_𝑆01≥1−(𝑘−1)𝜇𝛿_𝑆0²₂−𝜇𝛿_𝑆0²₁≥1−(𝑘−1)𝜇𝛿_𝑆0²₂−𝜇𝑘𝛿_𝑆0²₂≥1−(2𝑘−1)𝜇𝛿_𝑆0²₂, (33) where the second inequality applies the facts (A) and (B).
On the other hand, it follows from the fact (A) that 𝐴𝛿_𝑆0²₂≤1+(𝑘−1)𝜇𝛿_𝑆0²₂. (34)
Note that ‖𝐴𝛿‖₂=‖𝐴(𝑤−𝑤)‖₂≤𝐴𝑤−𝑦₂+𝐴𝑤−𝑦₂≤2𝜀. (35)
From Proposition 1 and the fact that |𝛿₍₁₎|≥|𝛿₍₂₎|≥⋯≥|𝛿_(𝑘+1)|≥|𝛿_(𝑘+2)|≥⋯, we get 𝛿_𝑆^𝑐₀²₂≤𝛿_𝑆^𝑐₀₁𝛿_𝑆01𝑘≤𝛿_𝑆0²₁𝑘≤𝛿_𝑆0²₂, (36) which implies ‖𝛿‖²₂=𝛿_𝑆0²₂+𝛿_𝑆^𝑐₀²₂≤2𝛿_𝑆0²₂. (37)
Putting them together, we get ‖𝛿‖₂≤√2𝛿_𝑆02≤√2𝐴𝛿,𝐴𝛿_𝑆01−(2𝑘−1)𝜇𝛿_𝑆02≤√2‖𝐴𝛿‖₂𝐴𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02≤221+(𝑘−1)𝜇𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02𝜀=221+(𝑘−1)𝜇1−(2𝑘−1)𝜇𝜀. (38)

Proof of Theorem 3. Note that from the fact (A) and the first part of the proof of Theorem 2, we have 𝐴𝛿,𝐴𝛿_𝑆0≥1−(2𝑘−1)𝜇𝛿_𝑆0²₂. (39) On the other hand, we also obtain the following relation: 𝐴𝛿,𝐴𝛿_𝑆0=𝐴𝛿_𝑆0^𝑇≤𝐴𝐴𝛿_𝑇𝐴𝛿_∞𝛿_𝑆01≤2𝜀√𝑘𝛿_𝑆02, (40)
We get, together with them, that 𝛿_𝑆02≤2𝜀√𝑘()𝜇.1−2𝑘−1 (41) Then ‖𝛿‖₂≤√2𝛿_𝑆02≤2√2𝑘1−(2𝑘−1)𝜇𝜀, (42) where the last second inequality uses (37).

Proof of Theorem 5. Let 𝛿=𝑤−𝑤 and 𝑇={1,2,…,𝑘}⊂{1,2,…,𝑛} be the support of 𝑤_max(𝑘). Following the notation and the first part in the proof of Theorem 2, we first give the following relation: 𝛿−𝛿_𝑆01≤𝛿_𝑆01+2𝑤_{−max(𝑘)1}. (43)
In fact, since ‖𝑤‖₁≥‖𝑤‖₁, we have ‖𝑤‖₁≥‖𝑤‖₁=‖𝑤+𝛿‖₁=𝑤_max(𝑘)+𝑤_−max(𝑘)+𝛿−𝛿_𝑆0+𝛿_𝑆01=𝑤_max(𝑘)+𝛿_𝑆01+𝑤_−max(𝑘)+𝛿−𝛿_𝑆01≥𝑤_max(𝑘)1−𝛿_𝑆01+𝛿−𝛿_𝑆01−𝑤_{−max(𝑘)1}. (44)
Since 𝑤=𝑤_max(𝑘)+𝑤_−max(𝑘), this yields 𝛿−𝛿_𝑆01≤𝛿_𝑆01+2𝑤_{−max(𝑘)1}. (45)
Note that 𝐴𝛿=𝐴𝑤−𝐴𝑤=0. (46)
So 0=𝐴𝛿,𝐴𝛿_𝑆0=𝐴𝛿_𝑆0,𝐴𝛿_𝑆0+𝐴𝛿−𝛿_𝑆0,𝐴𝛿_𝑆0≥1−(𝑘−1)𝜇𝛿_𝑆0²₂−𝜇𝛿−𝛿_𝑆01𝛿_𝑆01≥1−(𝑘−1)𝜇𝛿_𝑆0²₂−𝜇𝛿_𝑆01+2𝑤_{−max(𝑘)1}𝛿_𝑆01≥1−(2𝑘−1)𝜇𝛿_𝑆0²₂−2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆02, (47) where the last second inequality holds from  (43).
Then 𝛿₀₂≤2𝜇√𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (48)
From (37), we have that ‖𝛿‖₂≤√2𝛿₀₂≤2𝜇√2𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (49)
The proof is completed.

Proof of Theorem 6. From the proof of Theorem 5, we have 𝐴𝛿,𝐴𝛿_𝑆0≥()𝜇×𝛿1−2𝑘−1_𝑆022−2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆02. (50)
It follows from the fact (A) that 𝐴𝛿_𝑆0²₂≤1+(𝑘−1)𝜇𝛿_𝑆0²₂. (51)
Note that ‖𝐴𝛿‖₂=‖𝐴(𝑤−𝑤)‖₂≤𝐴𝑤−𝑦₂+𝐴𝑤−𝑦₂≤2𝜀. (52)
Together with them, we get 𝛿_𝑆02≤𝐴𝛿,𝐴𝛿_𝑆0+2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02≤‖𝐴𝛿‖₂𝐴𝛿_𝑆02+2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02≤2𝜀1+(𝑘−1)𝜇𝛿_𝑆02+2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02≤2𝜀1+(𝑘−1)𝜇+2𝜇√𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (53)
Then, from (37), ‖𝛿‖₂≤√2𝛿₀₂≤2𝜀21+(𝑘−1)𝜇+2𝜇√2𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (54)

Proof of Theorem 7. From the proof of Theorems 3 and 5, we get 𝛿_𝑆02≤𝐴𝛿,𝐴𝛿_𝑆0+2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02≤2𝜀√𝑘𝛿_𝑆02+2𝜇√𝑘𝑤_{−max(𝑘)1}𝛿_𝑆021−(2𝑘−1)𝜇𝛿_𝑆02=2𝜀√𝑘+2𝜇√𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (55) Then ‖𝛿‖₂≤√2𝛿₀₂≤2𝜀√2𝑘+2𝜇√2𝑘𝑤_{−max(𝑘)1}()𝜇.1−2𝑘−1 (56)