Abstract
This paper shows that discrete linear equations with Hilbert matrix operator, circulant matrix operator, conference matrix operator, banded matrix operator, TST matrix operator, and sparse matrix operator are ill-posed in the sense of Hadamard. Gauss least square method (GLSM), QR factorization method (QRFM), Cholesky decomposition method (CDM), and singular value decomposition (SVDM) failed to regularize these ill-posed problems. This paper introduces the eigenspace spectral regularization method (ESRM), which solves ill-posed discrete equations with Hilbert matrix operator, circulant matrix operator, conference matrix operator, and banded and sparse matrix operator. Unlike GLSM, QRFM, CDM, and SVDM, the ESRM regularizes such a system. In addition, the ESRM has a unique property, the norm of the eigenspace spectral matrix operator . Thus, the condition number of ESRM is bounded by unity, unlike the other regularization methods such as SVDM, GLSM, CDM, and QRFM.
1. Introduction
Many mathematical problems are classified as ill-posed problems. For instance, the discretization of linear ill-posed problems like the Fredholm integral equations of the first kind with a smooth kernel is ill-posed and hence referred to as a linear discrete ill-posed problem [1]. Moreover, image processing problems are mostly ill-posed when a blurred text or image is obtained. As a result, the image needs to be reconstructed to make a clear image. This study is concerned with the regularization of such linear discrete systems.
Given that a system of linear equations defined by where is a matrix operator in from a domain set into a codomain set , is a column vector in , and is also a column vector in , is well-posed in the sense of Hadamard if (1) is an surjective operator. Thus, the solution to the discrete equation (1) exists(2) is an injective operator. Thus, there is a unique solution to the discrete equation (1)(3)the inverse, , is continuous. Thus, the solution to equation (1) is stable (see authors in [2])
Although Hadamard provided the criteria for detecting an ill-posed problem, he did not provide any method for solving such a problem. The restoration of the well-posedness of a discrete system of the linear equation has received much attention since the beginning of the 20th century. Lavrentiev observed well-posed discrete equations as a system whose solution is sought on a compact subspace of a topological space which maps to another compact topological space. This type of well-posedness is generally referred to as conditionally well-posed discrete equations, [3].
Based on the varying definitions of well-posedness of discrete equations, Tikhonov defines the solution space as a compact set that is a subset of the Euclidean space. Still, there is no restriction on the data function, the vector that appears on the right-hand side of equation (1). The Tikhonov and Lavrentiev observations of well-posedness are generic definitions of Hadamard definitions. On the other hand, if a discrete equation violates one or more of the conditions stated above, then the equation is said to be ill-posed [4]. Equation (1) may have either no solution, or infinitely many solutions, or a unique solution.
Ill-posed discrete systems of linear equations are numerically unstable as they come with a large condition number. The instability of these matrix operators woefully compel the solution vector of the system of linear equations to be highly sensitive to perturbations in either the matrix operator or the right hand side of the system. This observation has been made by several researchers in the scientific community (for example, see [1, 5, 6]).
Ill-posed discrete equations can be regularized either by a direct approach or an iterative approach. Some direct regularization methods like Gauss least square method (GLSM) [7], QR factorization method (QRFM) [8], singular value decomposition method (SVDM) [9], and Cholesky decomposition method (CDM) [10]. In 1944, Tikhonov first constructed an iterative method that involves a regularization parameter for regularizing ill-posed discrete equations. There is no unique choice of the value of this parameter of even the same set of problems. In addition, the Tikhonov regularization method cannot restore the well-posedness of a discrete equation that does not have a solution.
Theorem 1 (Gauss least square method). If is an matrix with , then is nonsingular and the discrete equation (1) has a unique least squares solution given by See authors in [11].
The GLSM fails to solve the problem of stability and also performs poorly when solving underdetermined discrete equations. Though the GLSM may regularize the system, the operator maintains to be unstable, which fails to satisfy the stability condition for a well-posed system. In the work of [12, 13], the GLSM has been extended to meet the Lipchitz condition to satisfy the boundedness condition.
Theorem 2 (QR factorization method). Let be a matrix with entries in and linearly independent columns. Then, there exist a matrix and matrix , both having entries in , such that (1)(2), where is the transpose(3) is a nonsingular upper triangular matrix with (4)The columns of are an orthonormal basis for (5) and are unique (see author in [14])
In the work of [15], the QR decomposition has been merged with the Cholesky decomposition method to provide a faster algorithm in computations. The matrix operator , the upper triangular matrix, is closed in . We can see that Theorem 2 dwells on the existence of the solution to a discrete equation. QRFM restores only the existence and the uniqueness of the discrete system of linear equations. Also, the QRFM requires the matrix operator of the discrete equation given to be a linearly independent matrix operator (see author in [16]).
Theorem 3 (singular value decomposition method). Let be an matrix. There exist an integer , an unitary matrix , an unitary matrix , and an matrix , all of whose entries are except , such that See authors in [17].
In the work of [18], the SVDM has been applied in a correspondence analysis using R. On the other hand, the SVDM does not produce stable results dealing with discrete equations with or infinitely many solutions or no solution. The SVDM mostly has problems with the loss of small singular values, thereby giving undesirable answers in a functional space (see authors in [8, 19]).
Theorem 4 (Cholesky factorization method). Given a symmetric positive definite matrix , there exists a lower triangular matrix such that , where is the lower triangular matrix (Cholesky factor) and is the Cholesky factorization of (see author in [20]).
The Cholesky decomposition does not have a unique Cholesky decomposition for matrices that are not Hermitian and positive definite. Also, the CDM requires the matrix operator to be a symmetric positive definite matrix operator [10, 21]. This, therefore, suffices that the Cholesky method is unable to regularize some matrix operators and hence may still be ill-posed in the sense of Hadamard.
All these regularization methods in restoring the well-posedness of the ill-posed discrete equation (1) fail to restore the existence, uniqueness, and stability conditions of well-posedness of a discrete system of linear equation [(7)]. In a worse case, these methods struggle to resolve the well-posedness of the ill-posed discrete system of linear equations with a sparse matrix operator. In numerical analysis, matrix operators with large condition numbers are prone to significant numerical errors and also very sensitive to variations in either the right-hand side or even the matrix operator of equation (1) (see authors in [5]).
In addition, the GLSM, QRM, SVDM, and CDM fail to resolve the stability of the ill-posed discrete system of linear equation with Hilbert matrix, circulant matrix, conferences matrix, and the banded matrix. Since, after the applications of these methods to the discrete equation, the regularized matrix operator is mostly nearly singular and generally attributed to large condition numbers, the unstable solutions make these existing methods not reliable for approximating the solution of discrete equations.
In the area of application, the GLSM, QRFM, SVDM, and CDM have not been much consistently used to regularize an ill-posed discrete equation with perturbed right-hand side in equation (1) (see authors in [4]). Notwithstanding this, the populous matrix operators like the Hilbert matrix operator, conference matrix operator, circulant matrix operator, and banded matrix operator occurring in a discrete equation have not been looked at and talked about their regularization process vis-a-vis regularization method. The sparse matrix operator arising in a discrete equation commonly studied by many authors across the globe, when solved with SVDM, GLSM, CDM, and QRFM, does not yield the desired solution in functional space.
To overcome the myriad of ill-posed discrete equations raised in regularization, which solves these problems that are paramount, a method devoid of computational errors or minor computational errors is exciting and must be the focal point of regularizing discrete equations. This paper seeks to introduce a new regularization method, “the eigenspace spectral regularization method (ESRM),” for solving ill-posed discrete equations and check how efficient the ESRM is compared to the existing regularization methods.
In this paper, Section 1 contains a background to ill-posed problems. Also, in Section 2, we present the drawbacks of GLSM, QRFM, SVDM, and CDM when applied in regularizing discrete equations. Section 3 introduced the eigenspace spectral regularization method (ESRM) and used it to normalize discrete equations in which the GLSM, QRFM, SVDM, and CDM cannot solve. In addition, we also compare ESRM with GLSM, QRFM, SVDM, and CDM. Finally, Section 4 contains the conclusion of this paper.
2. The Ill-Posed Discrete System of Linear Equations and Drawbacks of Some Existing Methods
This section highlights definitions relevant to establishing ill-posed discrete equations and the regularization of such ill-posed discrete equations. Here, rigorous proofs for the ill-posed discrete equation are provided.
Several discrete systems of linear equations are discussed here. We give examples of ill-posedness of the discrete equations in the Hadamard sense.
Example 1. Consider the discrete equations below:
To determine if there is a solution to equation (4), we compute its augmented matrix as and the corresponding reduced row-echelon form is given also as
By the Rouche-Capelli theorem, the discrete equation (4) does not have a solution in since the rank of the matrix , which is less than the rank of the extended matrix, . Hence, the discrete equation (2) is ill-posed in the sense of Hadamard.
Example 2. Also, consider the discrete equation
Similarly, the discrete equation in (7) does not have a solution in because the rank of the matrix is , which is less than the rank of the extended matrix. Hence, the discrete equation (7) is ill-posed in the sense of Hadamard.
Example 3. Consider the discrete equation below:
It is also confirmed that the discrete equation in (8) has no solution in , thus, it is classified as an ill-posed discrete system of linear equations.
Example 4. Moreover, the discrete system of linear equation (9) below has its reduced row-echelon computed as
It was observed that the unknown variable is arbitrary. This implies that the discrete equation in (5) has an infinitely many solution. Hence, the discrete equation in (9) is also classified as ill-posed.
Example 5. Similar observations were made considering the discrete equation in (11) which has no unique solution in . Thus, it is classified as an ill-posed discrete system of linear equation.
Example 6. Again, we consider the discrete system of linear equation with a Toeplitz, Symmetric, and Tridiagonal (TST) matrix operator:
The augmented matrix for equation (12) is given as
and the corresponding reduced row-echelon form is given also as
It was observed that the system of linear equation (12) with a TST matrix operator is injective since both the rank of the matrix operator and the rank of the extended matrix is . Notwithstanding this, the inverse TST matrix operator is not continuous since . Therefore, the system of linear equation with a TST operator is classified as an ill-posed discrete system of linear equation.
Example 7. On the other hand, it was observed that the discrete equation with Hilbert matrix operator in equation (15) yields a unique solution but not stable since it has . Hence, the discrete equation is ill-posed in the sense of Hadamard.
Example 8. As seen in the discrete equation (15), similar observations were made given the discrete equation in (16) which has a banded matrix operator:
The discrete equation (15) yields an unstable solution since it has . Hence, the discrete equation is ill-posed in the sense of Hadamard.
Example 9. Also, the discrete equation with a conference matrix operator produces an unstable solution vector since it has . This therefore makes the discrete system of linear equation in (17) classified as an ill-posed problem.
Example 10. Similarly, the discrete system of linear equation in (18) which has a sparse matrix operator produced an unstable solution as . Thus, it is classified as an ill-posed problem.
Example 11. Finally, the discrete system of linear equation with a circulant matrix operator resulted in similar conclusions as observed in the discrete system of linear equations (15), (15), (17), and (18).
2.1. Drawbacks of Existing Methods of Regularizing Ill-Posed Discrete Equations
In this subsection, the shortcomings of GLSM, QRFM, SVDM, and CDM are highlighted. Though the abovementioned regularization methods are used in solving ill-posed discrete system of linear equations, they pose serious limitations.
2.1.1. Gauss Least Square Method
The GLSM is applied to the ill-posed discrete system of linear equations discussed in Section 2. Applying the GLSM to the discrete equation (4), it was observed that the GLSM operator yields a unique solution, that is, , on the grounds that , thus, nonsingular. On the other hand, , which is far from one. This implies that the solution of the discrete equation in (2) is not stable when solved using the GLSM. Hence, the GLSM fails to restore the well-posedness of the discrete equation in (4).
Also applying the GLSM to the discrete equation (7), the existence and uniqueness conditions of well-posedness are restored. Notwithstanding this, and also,
It was observed that the GLSM operator’s determinant is nearly degenerate. This implies instability of the solution of the discrete equation in (7) when solved by GLSM. Hence, the GLSM failed to regularize the discrete system of linear equation to be homomorphic.
Similarly, the discrete system of linear equation in (8) yields and also,
Thus, the GLSM could not restore the second and third conditions of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (8).
The GLSM as seen in the others fails to regularize the ill-posed discrete system of linear equation (9) since it yields
Thus, an unstable solution which is sensitive to perturbations since the condition number is far from unity.
Similar results were observed as the GLSM was applied to the discrete system of linear equation (11) as and also,
Thus, it fails to restore the unique condition of well-posedness, and again, the condition number is huge and hence fails to meet the third requirement of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (11).
It was further observed that the GLSM performed poorly in regularizing the ill-posed system of linear equation as the GLSM matrix operator yielded
Thus, the GLSM fails to regularize the ill-posed system of linear equation(12).
Also, applying the GLSM to the discrete system of linear equation (15), it yielded
Thus, a significant condition number fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (15).
Again, the GLSM when applied to the discrete system of linear equation produced which is far from unity and, thus, fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (16).
Considering also the discrete system of linear equation (17), it was observed that which is, thus, very sensitive to perturbations and fails to satisfy the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (17).
Again, in equation (18), the is very large and thus fails to satisfy the third condition of well-posedness. Thus, the GLSM fails to regularize the discrete equation in (18).
Finally, given the discrete equation (19), it is also observed that thus failing to satisfy the stability condition of well-posedness. Thus, the GLSM could not regularize the discrete equation (19).
2.1.2. QR Factorization Method
In this subsection, the concentration is on the regularization of discrete equations discussed in the previous section using the QRFM.
Applying the QRFM to the discrete system of linear equation, we observed that and therefore not feasible to proceed with this method. Hence, the QRFM fails to regularize the discrete equation in (4). Similar observations were made when applying the QRFM to the discrete equations in (7), (8), and (11).
On the other hand, the QRFM when applied to the discrete system of linear equation (9) yields which produces the unique solution , and . However, the solution is unstable since
Hence, QRFM could not regularize the discrete equation (9) in the sense of Hadamard.
Again, the QRFM matrix operator when applied to the ill-posed system of linear equation (12) given as was observed to be unstable as it reported a condition number of
Hence, the QRFM fails to regularize the ill-posed discrete system of linear equation (12) to be homeomorphic.
Similar trends are observed on the stabilities of solutions of discrete equations in (15), (16), (17), (18), and (19) when solved by the QRFM. The QRFM unequivocally fails to regularize the discrete system of linear equations in (4) to (18) from the analysis above.
2.1.3. Cholesky Decomposition Method
Also, the CDM is applied to the ill-posed discrete system of linear equations in order to make them homomorphic. Unfortunately, the discrete system of linear equation (4) could not be regularized by the CDM since equation (4) fails to meet the positive definite condition of CDM. Similar limitations were encountered when applying the CDM to the discrete equations (7), (8), (9), (11), (12), (16), (17), (18), and (19).
However, applying the CDM to the discrete system of linear equation (15) yields the matrix operator
It was observed that the CDM matrix operator yields a unique solution but very unstable since
Therefore, the discrete system of linear equation (15) could not be regularized using the CDM.
2.1.4. Singular Value Decomposition
The SVDM presents similar results; thus, it resolves the existence and uniqueness conditions of well-posedness in the discrete equations (4)–(19) but fails to satisfy the stability condition of well-posedness.
So far, these preliminary results signify that all of them, GLSM, SVDM, QRFM, and CDM, fail to restore the existence, uniqueness, and stability conditions of well-posedness in the discrete system of linear equations discussed in Section 2. It was also observed that the GLSM, SVDM, QRFM, and CDM all perform poorly in regularizing discrete system of linear equations with Hilbert, conference, circulant, banded, and sparse matrix operators.
3. Main Results
In this section, the Eigenspace Spectral Regularization Method (ESRM) is introduced and also proved. The ESRM is then applied to regularize the discrete equations in (4)–(18). Detailed discussions on how the ESRM restores the existence, uniqueness, and stability conditions of well-posedness in the ill-posed discrete system of linear equations are illustrated.
Theorem 5 (the eigenspace spectral regularization method). Suppose the discrete equation (1) where is the matrix operator, is a column vector in and is also a column vector in . The eigenspace spectral matrix operator eigenspace of , where . Then, a discrete equation in (1) has a stable unique solution given by where is the spectrum of
Proof. Given a discrete equation (1), set eigenspace of and the corresponding spectrumof , where . Replacing and in equation (1) by and , respectively, we have Multiplying both sides of the above equation by its adjoint operator of yields Since the columns of are linearly independent, thus, , it implies that exists. This suffices that implies This is a stable unique solution. In matrix theory , is the transpose of the matrix operator .
As indicated in the proof above, the ESRM is able to restore the existence, uniqueness, and stability conditions of well-posedness in the ill-posed discrete system of linear equations. We thus apply the ESRM to the ill-posed discrete system of equations discussed in Section 2 above. Note that all calculations were done by using MATLAB.
Firstly, applying the ESRM to the discrete equation (4), we compute which yields the ESRM matrix operator as and also the right hand side as
We thus compute which yields the unique solution vector since the determinant of the ESRM matrix operator is nonzero.
Also, the condition number of the ESRM matrix operator is computed as which is, thus, a stable solution which is not sensitive to perturbations. Hence, all three conditions of well-posedness are satisfied by the ESRM. Therefore, by applying the ESRM to the discrete system of linear equation (4) yields a homomorphic system.
Also, the ESRM performed better when applied to the discrete equation (7). The ESRM matrix operator was computed as and also the measured data was evaluated as
The ESRM again yields the unique solution since, which is, thus, a nondegenerate system. Also, the condition number of the ESRM matrix operator was computed as
This implies that the above solution is stable. Hence, the ESRM regularizes a discrete equation in (7). Thus, all three conditions of well-posedness are met. Therefore, the solution of the discrete equation in (7) is well-posed using ESRM.
The ESRM also regularizes the discrete system of linear equation (8) when applied to it. The matrix operator was calculated as and the measured data also computed as
It was observed firstly that the ESRM satisfies the solvability condition of well-posedness as it yields the solution vector
In addition, the ESRM yields a unique solution as it produces a nondegenerate matrix operator with . Also, the condition number of the ESRM matrix operator is computed as
Thus, the ESRM produces a stable solution to the discrete system of linear equation (8). Hence, the discrete equation (8) is a well-posed system in the sense of Hadamard by the ESRM.
Similarly, applying the ESRM to the discrete equation (9) with the ESRM matrix operator yields a nondegenerate system, on the grounds that .
Also, which is, thus, a stable solution. Therefore, by the ESRM, the discrete equation (9) is a homeomorphic system of linear equation in the Hadamard sense.
Moreover, the ESRM matrix operator derived from the discrete system of linear equation (11) is computed as which yields ; thus, the discrete equation (11) has a unique solution. The condition number for the ESRM matrix operator was also calculated as which indicates that the unique solution is very stable. Hence, the discrete system of linear equation (11) is a homomorphic system.
Subsequently, the ESRM performed better than all other regularization methods discussed in this paper, as the ESRM matrix operator was observed to be continuous as . Also, the ESRM inverse operator yielded which is, thus, a continuous mapping, that is, a stable solution.
It was further observed that, by applying the ESRM to the discrete system of linear equation (12) with the ESRM matrix operator also yields a unique and stable solution since and
Hence, the discrete equation (12) results in a homeomorphic discrete equation after applying the ESRM.
Again, applying the ESRM to the discrete equation (15) produces the ESRM matrix operator
It was again observed that ; thus the discrete equation (15) has a unique solution. In addition,
Thus, the discrete equation (15) has a stable solution. Therefore, the discrete equation (15) is a well-posed discrete equation in the Hadamard sense.
In addition, applying the ESRM to the discrete equation (16), we obtain the ESRM matrix operator as
We further observed that the ESRM yields a unique and stable solution, thus, restoring all three conditions of well-posedness in the discrete system of linear equation (16).
The ESRM when applied to the discrete system of linear equation (17) which has a sparse matrix operator yields a nondegenerate ESRM matrix operator
It is again observed that implying that the discrete equation (17) has a unique solution. Furthermore, the condition number was computed as
Thus, the ESRM restores the existence, uniqueness, and stability conditions of well-posedness in the discrete system of linear equation (17).
Last but not least, applying the ESRM to the discrete equation (18), we obtain the ESRM matrix operator which yields a unique solution since
It was again observed that which is, thus, an inversely bounded solution. Hence, the discrete equation (18) has been regularized by the ESRM to be a homeomorphic system of linear equation in the sense of Hadamard.
3.1. Main Findings in This Paper
This section concentrates on the performance of the ESRM as against the GLSM, QRFM, SVDM, and CDM in regularizing the above-discussed discrete system of linear equations. The results are shown in “Tables 1(a)–1(c).”
4. Conclusion
We have shown that only the ESRM could regularize the discrete system of linear equations (4), (7), (8), (9), (11), (12), (15), (16), (17), (18), and (19), respectively. Thus, the ESRM restores the existence, uniqueness, and stability conditions of well-posedness in the sense of Hadamard. However, the existing methods of regularization: GLSM, QRFM, SVDM, and CDM, failed to regularize the discrete system of linear equations (4) to (19). In addition, we observed that ESRM has a unique property; thus, the norm of the ESRM matrix operator . Also, the condition number of ESRM is bounded by unity, unlike the other regularization methods such as SVDM, GLSM, CDM, and QRFM.
Data Availability
This study did not make use of data.
Conflicts of Interest
The authors whose names are listed in the following certify that they have no affiliations with or involvement in any organization or entity with any financial interest or nonfinancial interest in the subject matter or materials discussed in this manuscript: Fredrick Asenso Wireko, Benedict Barnes, Charles Sebil, and Joseph Ackora-Prah.
Acknowledgments
The results in this work have been submitted as a preprint at the ScienceGate (see [22]).