Rate of Convergence of Tikhonov Method of Regularization for Constrained Linear Equations with Operators Having Closed Ranges

Abstract

We derive the estimates of the rate of convergence of the Tikhonov method of regularization for a constrained operator linear equation. In case that the range of the corresponding operator is closed, the estimate is of the same order as the estimates for a linear equation without constraints.

1. Introduction

Let 𝐻 and 𝐹 be real Hilbert spaces, 𝐴∶𝐻↦𝐹—a linear bounded operator from 𝐻 to 𝐹, 𝑓∈𝐹—a fixed element, and 𝑈⊆𝐻—a closed convex set. We will deal with the equation 𝐴𝑢=𝑓,𝑢∈𝑈. (1) Even when the constraint 𝑢∈𝑈 is absent, this problem can be ill-posed; that is, it is possible that there is 𝑢 which is far from the set of solution of (1), such that 𝐴𝑢≈𝑓. The ill-posedness of the equation in case of infinite-dimensional spaces 𝐻 and 𝐹 obviously comes from the fact that the range 𝑅(𝐴)∶={𝐴𝑢∶𝑢∈𝐻} of the operator 𝐴 is nonclosed. However, if the operator 𝐴 is known only approximately, then this equation can be ill-posed even in case of 𝑅(𝐴)=𝑅(𝐴). In this case, in order to solve the given problem, one has to use methods of regularization, for example, the Tikhonov method of regularization (see [1, 2]). Note that the presence of the constraints significantly complicates all issues concerning the ill-posedness.

Let us suppose that the set 𝑈 is given by 𝑈={𝑢∈𝐻∶ℎ(𝑢)≤0}, (2) where ℎ∶𝐻↦𝑅 is a continuously differentiable convex function satisfying Lipschitz and Slater conditions: (∃𝐿>0)(∀𝑢,𝜐∈𝐻)ℎ^′(𝑢)−ℎ^′(𝜐‖‖∃)≤𝐿𝑢−𝜐,𝑢∈𝑈∶ℎ(𝑢)<0. (3)(4)

We also suppose that sets 𝑈_∗ and 𝑈_∞ of the solutions of (1), and the corresponding equation without constraint 𝐴𝑢=𝑓,𝑢∈𝐻, (5) are nonempty. In this case, the minimization problems ‖𝑢‖²↦inf,𝑢∈𝑈_∗,‖𝑢‖²↦inf,𝑢∈𝑈_∞ (6)(7) have unique solutions that will be denoted by 𝑢_∗ and 𝑢_∞ and called normal solutions of (1) and (5).

Let us present one prominent example of the equation with the previously mentioned constraint with infinite-dimensional spaces 𝐻 and/or 𝐹.

Given that 𝑇>0 and matrices 𝐶(⋅)=(𝑐_𝑖𝑗(⋅))_𝑚×𝑚, 𝐷(⋅)=(𝑑_𝑖𝑗(⋅))_𝑚×𝑞 and 𝐺(⋅)=(𝑔_𝑖𝑗(⋅))_𝑛×𝑚, whose elements 𝑐_𝑖𝑗(⋅), 𝑑_𝑖𝑗(⋅), and 𝑔_𝑖𝑗(⋅) are piecewise continuous functions on [0,𝑇]. Let us consider a system in the so-called state-variable (or state-space) form: 𝑥^′(𝑡)(𝑡)𝑥(𝑡)(𝑡)𝑢(𝑡)[](0)𝑦(𝑡)(𝑡)𝑥(𝑡),=𝐶+𝐷,𝑡∈0,𝑇,𝑥=0,=𝐺 (8) where 𝑢=𝑢(𝑡),𝑡∈[0,𝑇] is a control from the space 𝐿^𝑞₂[0,𝑇]. This system of control is usually analyzed under some constraints on the control or on the trajectory. For example, the problems, “given 𝑓∈𝑅^𝑚, find a control (⋅)=𝑢=𝑢∈𝑈𝑢∈𝐿^𝑞₂∶^𝑇₀𝑥^′(𝑡,𝑢(𝑡))²𝑑𝑡≤𝑟², (9) such that 𝑦(𝑇)=𝑦(𝑇;𝑢)=𝑓” is a problem of type (1), (2). Then, one has to solve the equation 𝐴𝑢=𝑓 on the set (𝑢)=𝑈=𝑢∈𝐻∶ℎ_𝑇0𝑥^′(𝑡,𝑢(𝑡))²𝑑𝑡−𝑟²≤0, (10) where 𝐴𝑢=𝑦(𝑇;𝑢)∶𝐻=𝐿^𝑞₂[0,𝑇]↦𝐹=𝑅^𝑛. (11)

For example, electrical networks with a finite number of interconnected resistors, capacitors, and inductors can be described in state-variable form (see [3]).

The problems of the type (1) in the literature are usually considered under the assumption that, instead of the exact operator 𝐴 and instead of the element 𝑓, one actually deals with their approximations 𝐴_𝜂∈ℒ(𝐻,𝐹),𝑓_𝛿∈𝐹, such that 𝐴−𝐴_𝜂≤𝜂,𝑓−𝑓_𝛿≤𝛿,𝐴_𝜂≤𝑎, (12) where 𝜂>0,𝛿>0 are small positive real numbers and 𝑎>0.

As for the approximations of the normal solutions 𝑢_∗ and 𝑢_∞, one can use [1, 4] the so-called Tikhonov regularized approximate solutions 𝑢_𝛼 and 𝜐_𝛼, that is, the unique solutions of the regularized problems find 𝑢_𝛼∈𝑈 such that 𝐴^∗_𝜂𝐴_𝜂𝑢_𝛼−𝐴^∗_𝜂𝑓_𝛿+𝛼𝑢_𝛼,𝑢−𝑢_𝛼𝐴≥0,∀𝑢∈𝑈,_∗𝜂𝐴_𝜂𝜐_𝛼+𝛼𝜐_𝛼=𝐴^∗_𝜂𝑓_𝛿. (13)(14)

Let us observe that the above variational inequality (13) is necessary and sufficient conditions for the minimization problem of the Tikhonov function 𝑇_𝛼(𝑢)=12𝐴_𝜂𝑢−𝑓_𝛿²+𝛼2‖𝑢‖² (15) on the set 𝑈 or on the space 𝐻. The unique point 𝑢_𝛼(=𝑢_{𝛼(𝛿,𝜂)}) of the minimum of the function 𝑇_𝛼 on the set 𝑈 is said to be a regularized solution of (1).

The main aim of this paper is to establish whether there exists a choice of a parameter of regularization 𝛼=𝛼(𝛿,𝜂) which implies the convergence rate 𝑂(𝛿+𝜂) of regularized solutions 𝑢_𝛼=𝑢_{𝛼(𝛿,𝜂)} to 𝑢_∗. Namely, in [5], such convergence was proved in case of (5) without constraints and under the assumption that 𝑅(𝐴)=𝑅(𝐴). Note also that even in case of 𝐻=𝑅², the rate of convergence 𝑢_{𝛼(𝛿,𝜂)}→𝑢_∗, depending on the boundary of the set 𝑈, can be arbitrarily slow (see [6]).

Consequently, even in case of 𝑅(𝐴)=𝑅(𝐴), only if we require additional conditions on ℎ and 𝑢_∗, we can eventually expect the convergence rate 𝑂(𝛿+𝜂) of regularized solutions 𝑢_𝛼 to normal solution 𝑢_∗. In [7], it was proved that if 𝑢_∞ and projection of ℎ^′(𝑢_∗) belong to 𝑅(𝐴^∗), then for 𝛼=𝑂(𝛿+𝜂) yields ‖𝑢_𝛼−𝑢_∗‖=𝑂(𝛿+𝜂)^1/2.

The estimates of the rate of convergence of regularized solutions 𝑢_𝛼 to the normal solution 𝑢_∗ have been obtained in [8], under the conditions related to the smoothness of the boundary of the set 𝑈 and of the properties of the projection of 𝑢_∗ onto linear subspace of codimension 1, which is parallel to the tangential plane to the boundary of 𝑈.

Let us emphasize that the results of this paper are inspired by the results from [8]. However, our results related to the rate of convergence of regularized solutions were obtained under somewhat different general conditions and under the assumptions that, instead of the exact operator 𝐴, and instead of the element 𝑓, we only know their approximations 𝐴_𝜂 and 𝑓_𝛿.

The paper is organized as follows.

In Section 2, we described Tikhonov method of regularization adapted to the case when the set of constraints is given by (2). This constraint is taken into account by Lagrange multipliers. Lemma 1 was proved under the conditions of the sourcewise representativity type, that is, under the assumption that 𝑢_∗ belongs to the projection of 𝑅(𝐴^∗) onto 𝑈 [2, 6, 8]. In this section, we also present some results concerning the properties of the operators 𝐴^∗_𝜂𝐴_𝜂 and 𝐴_𝜂𝐴^∗_𝜂.

In Section 3, we suppose that the parameter of regularization 𝛼 is chosen in such way that 𝛼=𝑂(𝛿+𝜂). Under this assumption, we derive the estimates of the rate of convergence of Lagrange multipliers of regularized problems. The main result of the paper is contained in Theorem 11 which refers to the rate of convergence of regularized solutions.

2. Constrained Tikhonov Regularization and Previous Results

By applying the Kuhn-Tucker theorem to the minimization problem 𝑇_𝛼(𝑢)=12𝐴_𝜂𝑢−𝑓_𝛿²+𝛼2‖𝑢‖²,ℎ(𝑢)≤0, (16) we obtain that, for its unique solution 𝑢_𝛼=𝑢_{𝛼(𝛿,𝜂)}, there is 𝜆_𝛼≥0 such that 𝐴^∗_𝜂𝐴_𝜂𝑢_𝛼−𝐴^∗_𝜂𝑓_𝛿+𝛼𝑢_𝛼+𝜆_𝛼ℎ^′𝑢_𝛼𝜆=0,_𝛼ℎ𝑢_𝛼=0. (17) Using the function 𝑔_𝛼(𝑡)=1/(𝑡+𝛼),𝑡≥0, (in what follows, we will use this notation) the first equality in (17) can be written as 𝑔_𝛼⁻¹𝐴^∗_𝜂𝐴_𝜂𝑢_𝛼−𝐴^∗_𝜂𝑓_𝛿+𝜆_𝛼ℎ^′𝑢_𝛼=0, (18) or as 𝑢_𝛼=𝑣_𝛼−𝜆_𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_𝛼, (19) where 𝑣_𝛼=𝑔_𝛼(𝐴^∗_𝜂𝐴_𝜂)𝐴^∗_𝜂𝑓_𝛿 is the point of the minimum of the Tikhonov function 𝑇_𝛼 on 𝐻.

In what follows, we assume that the parameter 𝛼 is chosen such that 𝛼=𝑂(𝛿+𝜂).

Note that if ℎ(𝑣_𝛼)≤0, then 𝑢_∗=𝑢_∞, and for such 𝛼, the corresponding estimates concerning the accuracy of the regularized solutions of the equation 𝐴𝑢=𝑓,  𝑢∈𝐻, can be found, for example, in [5]. (Note that in [5], page 100, it was proved that if 𝑅(𝐴)=𝑅(𝐴), then for 𝛼=𝑂(𝛿+𝜂), the following estimate holds: ‖𝑢_∞−𝑣_𝛼‖=𝑂(𝛿+𝜂)). For this reason, in what follows, we will suppose that ℎ(𝑣_𝛼)>0 for all sufficiently small 𝛿 and 𝜂. In this case, we have 𝜆_𝛼≠0 and, consequently, ℎ(𝑢_𝛼)=0 and ℎ^′(𝑢_𝛼)≠0 for all such 𝛿 and 𝜂. Therefore, bearing in mind that 𝑢_𝛼→𝑢_∗ as 𝛿,𝜂→0, we conclude that ℎ(𝑢_∗)=0. The Slater condition (4) then implies that 𝑢_∗ is not a point of minimum of ℎ, and, consequently, ℎ^′(𝑢_∗)≠0.

Now, as a consequence of the variational inequality (13), taking into account the equality (17) and 𝜆_𝛼>0, we obtain (∀𝑢∈𝑈)ℎ^′𝑢_𝛼,𝑢−𝑢_𝛼≤0. (20) From here, we have (∀𝑢∈𝑈)ℎ^′𝑢_∗,𝑢−𝑢_∗≤0. (21) Consequently, ℎ^′𝑢_∗,𝑢_𝛼−𝑢_∗≤0,ℎ^′𝑢_𝛼,𝑢_∗−𝑢_𝛼≤0. (22)

Further estimates of the rate of convergence of the regularized solutions are based on the next property of the normal solution 𝑢_∞∈𝑅(𝐴^∗)=𝑅(𝐴^∗𝐴), (23) that for the operator with closed range becomes 𝑢_∞∈𝑅𝐴^∗𝐴;thatis,𝑢_∞=𝑅𝐴^∗𝐴𝑤_∗,𝑤_∗∈𝐻. (24) As it was argued in [6] (see also [9]), this condition does not have the same meaning as it has in case of problem (6), because the normal solution 𝑢_∗ of the corresponding problem without constraints does not satisfy the condition 𝑢_∗∈𝑅(𝐴^∗𝐴). However, it is easy to prove that 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗)), (25) where by 𝜋_𝑋, we denote the operator of the metric projecting on the closed convex set 𝑋⊆𝐻.

Lemma 1. 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗))if and only if there exists 𝛾_∗≥0 such that 𝑢_∗=𝑢_∞−𝛾_∗(𝐼−𝜋_{𝑅(𝐴∗𝐴)})ℎ^′(𝑢_∗).

Proof. For any 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗)), there is 𝑒_∗∈𝑅(𝐴^∗) such that 𝑢_∗ is a solution of the minimization problem 12𝑢−𝑒_∗²⟶inf,ℎ(𝑢)≤0. (26) According to the Kuhn-Tucker theorem, there exists a real number 𝛾_∗≥0 such that 𝑢_∗−𝑒_∗+𝛾_∗ℎ^′(𝑢_∗)=0. From here, bearing in mind that 𝑒_∗∈𝑅(𝐴^∗), we obtain 𝐼−𝜋_{𝑅(𝐴∗)}𝑢_∗=−𝛾_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗. (27) Consequently, 𝑢_∗=𝑢_∞+𝐼−𝜋_{𝑅(𝐴∗)}𝑢_∗=𝑢_∞−𝛾_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗. (28)
Now, let us suppose that there exists 𝛾_∗≥0 such that 𝑢_∗=𝑢_∞−𝛾_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗. (29)
This equality can be written in the form of 𝑢_∗−𝑢_∞−𝛾_∗𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗=−𝛾_∗ℎ^′𝑢_∗. (30) Multiplying both sides of this equality by 𝑢−𝑢_∗ and bearing in mind (21), it comes out that 𝑢_∗−𝑢_∞+𝛾_∗𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗,𝑢−𝑢_∗=−𝛾_∗ℎ^′𝑢_∗,𝑢−𝑢_∗≥0,𝑢∈𝑈. (31) The last equality means that 𝑢_∗=𝜋_𝑈(𝑢_∞+𝛾_∗𝜋_{𝑅(𝐴∗)}ℎ^′(𝑢_∗)). Having in mind that 𝑢_∞+𝛾_∗𝜋_{𝑅(𝐴∗)}ℎ^′(𝑢_∗)∈𝑅(𝐴^∗), it follows that 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗)).

As an immediate consequence of this lemma, we have that if the range of the operator 𝐴∶𝐻→𝐹 is closed subspace of 𝐹, then 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗)) if and only if there is 𝛾_∗≥0such that 𝑢_∗=𝑢_∞−𝛾_∗(𝐼−𝜋_𝑅(𝐴^∗₎)ℎ^′(𝑢_∗).

Finally, let us remark that it is easy to construct an example of convex set 𝑈⊆𝑅² and operator 𝐴∶𝑅²↦𝑅² such that 𝑢_∗∉𝜋_𝑈(𝑅(𝐴^∗)).

Further, we will continue with three lemmas related to properties of the operators 𝐴^∗_𝜂𝐴_𝜂 and 𝐴_𝜂𝐴^∗_𝜂, whose proofs can be found in [5], pages 99, 153, 154, and 156.

Lemma 2. If ‖𝐴−𝐴_𝜂‖≤𝜂, then 𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗_𝜂𝐴_𝜂−𝐼ℎ⟶0𝑎𝑠𝛼⟶0,𝜂⟶0 (32) for all ℎ∈𝑅(𝐴^∗).

Lemma 3. (a) The range of the bounded linear operator 𝐴∶𝐻→𝐹 is closed subspace of 𝐻 if and only if 𝑚_𝐴∶=inf{‖𝐴𝑢‖∶‖𝑢‖=1,𝑢⟂Ker𝐴}>0. (33)
(b) If the bounded linear operators 𝐴∶𝐻↦𝐹 and 𝐴_𝜂∶𝐻↦𝐹 satisfy the condition 𝑚_𝐴>0,𝐴−𝐴_𝜂≤𝜂,0<2𝜂<𝑚_𝐴, (34) then 𝜎(𝐴^∗_𝜂𝐴_𝜂),𝜎(𝐴_𝜂𝐴^∗_𝜂)⊆[0,𝜂²]⋃[(𝑚_𝐴−𝜂)²,‖𝐴_𝜂‖²], where 𝜎(𝐴^∗_𝜂𝐴_𝜂) and 𝜎(𝐴_𝜂𝐴^∗_𝜂) are the spectrums of the operators 𝐴^∗_𝜂𝐴_𝜂, and 𝐴_𝜂𝐴^∗_𝜂.

By 𝜋_𝜂, we will denote the operator of orthogonal projecting of the space 𝐻 on invariant subspace of the operator 𝐴^∗_𝜂𝐴_𝜂 that corresponds to the part of the spectrum belonging to [(𝑚_𝐴−𝜂)²,‖𝐴_𝜂‖²]. Then, 𝐼−𝜋_𝜂 is the operator of orthogonal projecting of the space 𝐻 on invariant subspace of the operator 𝐴^∗_𝜂𝐴_𝜂 that corresponds to the part of the spectrum belonging to [0,𝜂²].

Lemma 4. If the conditions (34) are satisfied, then, 𝜋_𝑅(𝐴^∗₎−𝜋_𝜂≤𝜂𝑚_𝐴,𝜋−𝜂_𝜂𝐼−𝜋_{𝑅(𝐴∗)}≤𝜂𝑚_𝐴,𝐴−𝜂_𝜂𝐼−𝜋_𝜂≤𝜂. (35)

As the consequences of the previous lemmas, it is easy to prove the following estimates.

Lemma 5. Let the conditions (34) be satisfied. Then 𝜋_𝜂𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂≤1𝑚_𝐴−𝜂². (36) If, additionally, the parameter 𝛼 of regularization is chosen such that 𝛼=𝑂(𝛿+𝜂), then the following estimates hold for all 𝑥: 𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗_𝜂1≤max1;𝑚_𝐴,−𝜂𝐼−𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐼−𝜋_𝜂𝑥1≤max1;𝑚_𝐴−𝜂𝐼−𝜋_𝜂𝑥𝜂,𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝑥−𝐼−𝜋_𝜂𝑥≤1𝑚_𝐴−𝜂²1+max1;𝑚_𝐴‖𝑥‖.−𝜂𝛿+𝜂 (37)(38)(39)

The last result that we will cite in this section concerns the rate of convergence of the Tikhonov regularization of the equation 𝐴𝑢=𝑓, 𝑢∈𝐻, and its proof can be found in [4] and also in [5].

Theorem 6. If the conditions (12) are satisfied and the parameter 𝛼=𝛼(𝛿,𝜂) is chosen such that 𝛼(𝛿,𝜂)→0 and 𝛿+𝜂/𝛼→0 as 𝛿→0,𝜂→0, then the regularized solutions 𝑣_{𝛼(𝛿,𝜂)} of (14) converge to the normal solution 𝑢_∞ of (5) as 𝛿→0,𝜂→0.
If, additionally, the range of the operator 𝐴 is closed, then for 𝛼=𝑂(𝛿+𝜂), the following estimate holds: 𝑣_𝛼−𝑢_∞=𝑂𝛿+𝜂. (40)

3. Convergence and Rate of Convergence

In order to derive an estimate for the solutions of the equation 𝐴𝑢=𝑓,𝑢∈𝑈⊆𝐻, we will begin this section with lemmas concerning the properties of Lagrange multipliers 𝜆_𝛼 for problem (17), that will be used in the proof of the estimates of the rate of convergence. Note that throughout the section we will suppose that the parameter of the regularization 𝛼 is chosen such that 𝛼=𝑂(𝛿+𝜂).

Lemma 7. If the conditions (34), (12) are satisfied and (𝐼−𝜋)ℎ^′(𝑢_∗)≠0, then for 𝛼=𝑂(𝛿+𝜂), the following estimate is true: 𝜆_𝛼𝛼−𝛾_∗=𝑂𝛿+𝜂+𝑢_∗−𝑢_𝛼, (41) where 𝜆_𝛼 and 𝛾_∗ are Lagrange multipliers from (17) and Lemma 2.

Proof. By (39) and Lemma 4, from the equality 𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗−𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗=𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗−𝐼−𝜋_𝜂ℎ^′𝑢_∗+𝜋_{𝑅(𝐴∗)}−𝜋_𝜂ℎ^′𝑢_∗, (42) we get that 𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗⟶𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗as𝛿,𝜂⟶0,𝛼=𝑂𝛿+𝜂. (43) Furthermore, since 𝑢_𝛼→𝑢_∗,ℎ^′(𝑢_𝛼)→ℎ^′(𝑢_∗), we have 𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼⟶𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗²≠0as𝛼=𝑂𝛿+𝜂⟶0. (44) Consequently, 𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼≠0 (45) for all sufficiently small 𝛿 and 𝜂. Multiplying both sides of (19) by ℎ^′(𝑢_∗) and taking into consideration the last inequality, we obtain 𝜆_𝛼𝛼=ℎ^′𝑢_∗,𝑣_𝛼−𝑢_𝛼𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼. (46) Now, multiplying the equality 𝑢_∗=𝑢_∞−𝛾_∗(𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗) (see Lemma 1) by (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗) and taking into account that 𝑢_∞−𝑢_∗∈Ker𝐴, we find that 𝛾_∗=ℎ^′𝑢_∗,𝑢_∞−𝑢_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗². (47) In this way, we obtain the equality 𝛾_∗−𝜆_𝛼𝛼=ℎ^′𝑢_∗,𝑢_∞−𝑣_𝛼+ℎ^′𝑢_∗,𝑢_𝛼−𝑢_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗²+ℎ^′𝑢_∗,𝑣_𝛼−𝑢_𝛼𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗²𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼×𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼−𝐼−𝜋_{𝑅(𝐴∗)}𝑢_∗². (48) Let us observe that 𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼−𝐼−𝜋_{𝑅(𝐴∗)}𝑢_∗²=𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼−ℎ^′𝑢_∗+𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝑔^′𝑢_∗−𝐼−𝜋_𝜂ℎ^′𝑢_∗,𝑔^′𝑢_∗+𝜋_{𝑅(𝐴∗)}−𝜋_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_∗. (49) Now, having in mind condition (3), estimate (39), Lemma 4, and Theorem 6, we get 𝛾_∗−𝜆_𝛼𝛼=𝑂𝛿+𝜂+𝑢_∗−𝑢_𝛼. (50)

In the following theorem, we will present one result concerning the estimates of the discrepancies ‖𝐴𝑢_𝛼−𝑓‖ and ‖𝐴_𝜂𝑢_𝛼−𝑓_𝛿‖.

Theorem 8. If the conditions (34) and (12) are satisfied and 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗)), then for 𝛼=𝑂(𝛿+𝜂), the following estimates hold: 𝐴𝑢_∗−𝑢_𝛼=𝑂𝛿+𝜂,𝐴_𝜂𝑢_∗−𝑢_𝛼,𝐴=𝑂𝛿+𝜂_𝜂𝑢_𝛼−𝑓_𝛿𝛼=𝑂(1). (51)

Proof. First, let us consider the case when (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)=0. Then, based on (19) and Lemma 1, we have that 𝑢_∗=𝑢_∞ and 𝑢_∗−𝑢_𝛼=𝑢_∞−𝑣_𝛼+𝜆_𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_𝛼. (52) Multiplying both sides of this equality by 𝑔_𝛼⁻¹(𝐴^∗_𝜂𝐴_𝜂)(𝑢_∗−𝑢_𝛼) and taking into account (22), we obtain 𝑔_𝛼⁻¹𝐴^∗_𝜂𝐴_𝜂𝑢_∗−𝑢_𝛼,𝑢_∗−𝑢_𝛼≤𝑔_𝛼⁻¹𝐴^∗_𝜂𝐴_𝜂𝑢_∞−𝑣_𝛼,𝑢_∗−𝑢_𝛼. (53) The last inequality can be written in the form 𝐴_𝜂𝑢_∗−𝑢_𝛼²+𝛼𝑢_∗−𝑢_𝛼²≤𝐴_𝜂𝑢_∞−𝑣_𝛼²+𝛼𝑢_∞−𝑣_𝛼². (54) In this case, from here and from Theorem 6, it follows that 𝐴_𝜂𝑢_∗−𝑢_𝛼,𝐴𝑢=𝑂𝛿+𝜂_∗−𝑢_𝛼≤𝐴−𝐴_𝜂𝑢_∗−𝑢_𝛼+𝐴_𝜂𝑢_∗−𝑢_𝛼=𝑂𝛿+𝜂. (55)
Now, let us consider the case of (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)≠0. Then, we have that 𝑢_∗=𝑢_∞−𝛾_∗(𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗); that is 𝑢_∗−𝑢_𝛼=𝑢_∞−𝑣_𝛼+𝜆_𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_𝛼−𝛾_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗, (56) from where we obtain that 𝐴_𝜂𝑢_∗−𝑢_𝛼=𝐴_𝜂𝑢_∞−𝑣_𝛼𝜆+𝛼^𝛼𝛼𝐴_𝜂𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_𝛼−𝛾_∗𝐴_𝜂×−𝐴𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗. (57) From here, by Theorem 6, Lemma 5 and having in mind (37) and condition (12), we obtain the estimate 𝐴_𝜂𝑢_∗−𝑢_𝛼,𝐴𝑢=𝑂𝛿+𝜂_∗−𝑢_𝛼=𝑂𝛿+𝜂. (58)
The third estimate in the theorem is an immediate consequence of (12) and the equality 𝐴_𝜂𝑢_𝛼−𝑓_𝛿=𝐴_𝜂𝑢_𝛼−𝑢_∗+𝐴_𝜂−𝐴𝑢_∗+𝑓−𝑓_𝛿. (59)

Before presenting the proof of the main result, we will prove two lemmas: the first regarding the convergence of Lagrange multipliers 𝜆_𝛼 from equality (17) and the second concering the projection of 𝑢_∗−𝑢_𝛼 on the subspace 𝑅(𝐴^∗𝐴).

Lemma 9. Suppose the conditions (34) and (12) are satisfied and 𝑢_∗∈𝜋_𝑈(𝑅(𝐴^∗), (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)=0. Then, for 𝛼=𝑂(𝛿+𝜂) one has 𝜆_𝛼𝛼=𝑂(1). (60)

Proof. From (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)=0, it follows that 𝑢_∗=𝑢_∞ and ℎ^′𝑢_∗=𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗∈𝑅𝐴^∗𝐴. (61) Then, there is 𝑒∈𝑅(𝐴^∗𝐴) such that ℎ^′(𝑢_∗)=𝐴^∗𝐴𝑒 and consequently 𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗=𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗𝐴𝑒=𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝑒+𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗_𝜂𝐴_𝜂−𝐼ℎ+ℎ. (62) Adding and subtracting (𝐼−𝜋_𝜂)((𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂)/𝛼)𝑒 on the right side of this equality and then multiplying both sides by ℎ^′(𝑢_𝛼), we obtain 𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼=𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝛼𝑒−𝐼−𝜋_𝜂𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝛼𝑒,ℎ^′𝑢_𝛼+𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝛼𝑒,𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_𝛼+𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗_𝜂𝐴_𝜂−𝐼𝑒,ℎ^′𝑢_𝛼+𝑒,ℎ^′𝑢_𝛼. (63) Observe that ‖(𝐼−𝜋_{𝑅(𝐴∗)})((𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂)/𝛼)𝑒‖ remains bounded when 𝛿→0, 𝜂→0 and 𝛼=𝑂(𝛿+𝜂). Now, using (37), we have that 𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝛼𝑒−𝐼−𝜋_𝜂𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝛼𝑒=𝑂𝐼−𝜋_𝜂𝐴^∗𝐴−𝐴^∗_𝜂𝐴_𝜂𝛼𝑒𝛿+𝜂⟶0. (64) Besides, (𝑔_𝛼(𝐴^∗_𝜂𝐴_𝜂)𝐴^∗_𝜂𝐴_𝜂−𝐼)𝑒→0 (Lemma 2) and 𝐼−𝜋_𝜂ℎ^′𝑢_𝛼⟶𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗⟶0as𝛼=𝑂𝛿+𝜂⟶0. (65) Now, from (63), it follows that lim_𝛼→0𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼=𝑒,ℎ^′𝑢_∗=𝑒,𝐴^∗=‖‖𝐴𝑒𝐴𝑒²≠0. (66) It means that ⟨𝑔_𝛼(𝐴^∗_𝜂𝐴_𝜂)ℎ^′(𝑢_∗),ℎ^′(𝑢_𝛼)⟩≠0 for all sufficiently small 𝛿 and 𝜂. Multiplying (19) by ℎ^′(𝑢_∗) and taking into account the last inequality, we will obtain 𝜆_𝛼𝛼=1𝛼ℎ^′𝑢_∗,𝑣_𝛼−𝑢_𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼. (67) If we replace ℎ^′(𝑢_∗) and 𝑢_∗ in the previous equality by 𝐴^∗𝐴𝑒 and 𝑢_∞, it becomes 𝜆_𝛼𝛼=1𝛼⟨𝐴𝑒,𝐴𝑣_𝛼−𝑢_∞+𝐴𝑢_∗−𝑢_𝛼⟩⟨𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_∗,ℎ^′𝑢_𝛼⟩. (68) Based on Theorem 8, it follows the boundness of ‖𝐴(𝑣_𝛼−𝑢_∞)‖/𝛼 and ‖𝐴(𝑢_∗−𝑢_𝛼)‖/𝛼 as 𝛼=𝑂(𝛿+𝜂)→0. Now, it is clear that 𝜆_𝛼/𝛼 also remains bounded as 𝛼=𝑂(𝛿+𝜂).

Lemma 10. Suppose the conditions (34) and (12) are satisfied. If 𝛼=𝑂(𝛿+𝜂), then ‖𝜋_{𝑅(𝐴∗)}(𝑢_∗−𝑢_𝛼)‖=𝑂(𝛿+𝛼).

Proof. From 𝑢_∗−𝑢_𝛼=𝑢_∞−𝑣_𝛼+𝜆_𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂ℎ^′𝑢_𝛼−𝛾_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗, (69) we have that 𝜋_𝜂𝑢_∗−𝑢_𝛼=𝜋_𝜂𝑢_∞−𝑤_𝛼+𝜆_𝛼𝛼𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝜋_𝜂ℎ^′𝑢_𝛼−𝛾_∗𝜋_𝜂𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗. (70) Consequently, 𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼=𝜋_{𝑅(𝐴∗)}−𝜋_𝜂×𝑢_∗−𝑢_𝛼+𝜋_𝜂𝑢_∞−𝑣_𝛼+𝜆_𝛼𝛼𝛼𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂𝜋_𝜂ℎ^′𝑢_𝛼−𝛾_∗𝜋_𝜂𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗. (71) The value 𝜆_𝛼/𝛼 remains bounded as the parameter 𝛼=𝑂(𝛿+𝜂) converges to zero, in case of (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)=0, as well as in case of (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)≠0. Now, for 𝛼=𝑂(𝛿+𝜂), the estimate 𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼=𝑂𝛿+𝜂 (72) follows from 𝜋_𝜂−𝜋_{𝑅(𝐴∗)}𝜂,𝑢=𝑂_∞−𝑣_𝛼,𝜋=𝑂𝛿+𝜂_𝜂𝐼−𝜋_{𝑅(𝐴∗)}𝜂,𝜋=𝑂_𝜂𝑔_𝛼𝐴^∗_𝜂𝐴_𝜂≤1𝑚_𝐴−𝜂². (73)

The following theorem is the main result of the paper.

Theorem 11. Suppose the conditions (12) and (34) are satisfied and 𝑢_∗∈𝜋_𝑅(𝐴^∗₎ and 𝛼=𝑂(𝛿+𝜂). Then, 𝑢_∗−𝑢_𝛼=𝑂𝛿+𝜂. (74)

Proof. From (17), we obtain the following equality: 𝐼−𝜋_{𝑅(𝐴∗)}𝑢_𝛼=1𝛼𝐼−𝜋_{𝑅(𝐴∗)}×𝐴^∗−𝐴^∗_𝜂𝑓_𝛿−𝐴_𝜂𝑢_𝛼−𝜆_𝛼𝛼𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_𝛼. (75) From here, taking into account the equality 𝑢_∗=𝑢_∞−𝛾_∗(𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗), we have 𝑢_∗−𝑢_𝛼=𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼+𝐼−𝜋_{𝑅(𝐴∗)}𝑢_∗−𝐼−𝜋_{𝑅(𝐴∗)}𝑢_𝛼=𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼+𝜆_𝛼𝛼𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_𝛼−𝛾_∗𝐼−𝜋_{𝑅(𝐴∗)}ℎ^′𝑢_∗+1𝛼𝐼−𝜋_{𝑅(𝐴∗)}×𝐴^∗−𝐴^∗_𝜂𝐴_𝜂𝑢_𝛼−𝑓_𝛿. (76) Multiplying both sides of this equality by 𝑢_∗−𝑢_𝛼 and taking into account (13), we obtain 𝑢_∗−𝑢_𝛼²=𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼+𝐼−𝜋_{𝑅(𝐴∗)}×𝐴^∗−𝐴^∗_𝜂𝐴_𝜂𝑢_𝛼−𝑓_𝛿𝛼,𝑢_∗−𝑢_𝛼+𝛾_∗ℎ^′𝑢_∗,𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼−𝜆_𝛼𝛼ℎ^′𝑢_𝛼,𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼. (77) Now, adding and subtracting (𝜆_𝛼/𝛼)⟨ℎ^′(𝑢_∗),𝜋_{𝑅(𝐴∗)}(𝑢_∗−𝑢_𝛼)⟩ on the right side of this equality, it becomes 𝑢_∗−𝑢_𝛼=𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼+𝐼−𝜋_{𝑅(𝐴∗)}×𝐴^∗_𝜂−𝐴^∗𝐴_𝜂𝑢_𝛼−𝑓_𝛿𝛼,𝑢_∗−𝑢_𝛼+𝛾_∗−𝜆_𝛼𝛼ℎ^′𝑢_∗,𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼+𝜆_𝛼𝛼ℎ^′𝑢_∗−ℎ^′𝑢_𝛼,𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼. (78)
Observe that in case of (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)=0, we can take 𝛾_∗=0. Then, 0≤⟨ℎ^′(𝑢_∗),𝑢_∗−𝑢_𝛼⟩=⟨ℎ^′(𝑢_∗),𝜋_{𝑅(𝐴∗)}(𝑢_∗−𝑢_𝛼)⟩. From here and (78), we obtain 𝑢_∗−𝑢_𝛼²≤𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼+𝐼−𝜋_{𝑅(𝐴∗)}×𝐴^∗_𝜂−𝐴^∗𝐴_𝜂𝑢_𝛼−𝑓_𝛿𝛼,𝑢_∗−𝑢_𝛼+𝜆_𝛼𝛼ℎ^′𝑢_∗−ℎ^′𝑢_𝛼,𝜋_{𝑅(𝐴∗)}𝑢_∗−𝑢_𝛼. (79) Finally, from here, by Lemmas 9 and 10, Theorem 8, and conditions (12) and (3), the estimate (74) follows.
In case of (𝐼−𝜋_{𝑅(𝐴∗)})ℎ^′(𝑢_∗)≠0, the estimate (74) is an immediate consequence of Lemmas 7 and 9, Theorem 8, and conditions (12).

Acknowledgments

The authors would like to thank the reviewers for their valuable suggestions and comments.