New Models for Solving Time-Varying LU Decomposition by Using ZNN Method and ZeaD Formulas
In this paper, by employing the Zhang neural network (ZNN) method, an effective continuous-time LU decomposition (CTLUD) model is firstly proposed, analyzed, and investigated for solving the time-varying LU decomposition problem. Then, for the convenience of digital hardware realization, this paper proposes three discrete-time models by using Euler, 4-instant Zhang et al. discretization (ZeaD), and 8-instant ZeaD formulas to discretize the proposed CTLUD model, respectively. Furthermore, the proposed models are used to perform the LU decomposition of three time-varying matrices with different dimensions. Results indicate that the proposed models are effective for solving the time-varying LU decomposition problem, and the 8-instant ZeaD LU decomposition model has the highest precision among the three discrete-time models.
The LU decomposition is generally described as decomposing a matrix into a unit lower triangular matrix (simply termed, ) and an upper triangular matrix (simply termed, ). Meanwhile, the matrix equals the product of and , that is, . This is also the origin of the name of LU decomposition. In numerical analysis, the LU decomposition is applied to solving linear systems, calculating the matrix determinant, and so on. Moreover, LU decomposition, as an important branch of matrix decompositions, also has many applications in science and engineering fields [1–6], such as in multidimensional frequency estimation , array structures , and power flow algorithm . In addition, the LU decomposition-based techniques play an important role in solving various engineering problems. For instance, in order to obtain a secure and robust watermarking algorithm, the LU decomposition was utilized in . Besides, the LU decomposition was employed to find the complex nonorthogonal joint diagonalization in . In , for fast circuit analysis, the parallel sparse LU decomposition method was used. Therefore, solving the LU decomposition problem is very meaningful.
For solving the LU decomposition problem, many algorithms or methods were proposed. For example, an LU-spatial decomposition method was proposed in . The method uses a partitioning scheme to compute the LU decomposition rather than the explicit inverse, which reduces the computational cost efficiently. A recursive LU decomposition algorithm was proposed in . The method can also save the computation time but needs to make geometrical modifications to the precomputed scatterer. What is more, as for the nonsingular matrix, a method to find the quasi-LU decomposition was proposed in . The method can obtain the quasi-LU decomposition of a matrix though the matrix does not has LU decomposition. However, those proposed algorithms or methods are not designed to find the LU decomposition of the time-varying matrix. Thus, for the time-varying matrix, those methods have no significant advantages to find the LU decomposition and may have lag errors. For LU decomposition of the time-varying matrix, there is little research while the time-varying problems are becoming more and more important [16–18]. Besides, an important application of time-varying LU decomposition is solving the angle-of-arrival (AoA) localization problem , which is consistent with the time-varying linear system and has been widely applied in various fields [20, 21]. The AoA localization problem can be formulated as . Thus, using a model or method (which can obtain the LU decomposition of the time-varying matrix in real time) can accelerate the solving progress of the AoA localization problem.
Due to the powerful capabilities in time-varying information processing, Zhang neural network (ZNN) method has been applied to solving various time-varying problems [16–18, 22–28]. For example, for finding real-time matrix square root, a finite-time convergent ZNN was proposed in . In order to solve the time-varying matrix inversion problem, a novel discrete-time ZNN was proposed in . Therefore, the ZNN method is employed to solve the time-varying LU decomposition problem in this paper.
In the process of applying the ZNN method to solving the time-varying problems [29–31], an error function is constructed firstly. Then, by using the ZNN design formula, the error function is forced to converge to zero. Furthermore, the continuous-time model for solving the original problem is obtained. In this paper, the corresponding model is termed the continuous-time LU decomposition (CTLUD) model. In addition, by employing three one-step forward finite difference formulas (i.e., Euler, 4-instant Zhang et al. discretization (ZeaD), and 8-instant ZeaD formulas) to discretize the proposed CTLUD model, three corresponding discrete-time models are obtained.
The remainder of this paper is composed of six sections. In Section 2, the problem formulation of the time-varying LU decomposition is presented. In Section 3, the design process of the CTLUD model is presented. In Section 4, three discretization formulas are presented and corresponding discrete-time models are obtained. In Section 5, the proposed CTLUD model is employed to perform LU decomposition of three time-varying matrices with different dimensions, and the corresponding results are shown. In Section 6, the experiment results of three discrete-time models are presented. In Section 7, this paper is concluded. Before ending the introduction part, the main contributions of this paper are recapped as follows:(1)Different from static LU decomposition problem analysis, this paper considers and analyzes the time-varying LU decomposition problem.(2)An effective CTLUD model is proposed by employing the ZNN method, Kronecker product, and vectorization techniques.(3)Three discrete-time LU decomposition models are obtained by employing three discretization formulas to discretize the proposed CTLUD model.(4)Experiment results substantiate that the proposed models are effective for solving the time-varying LU decomposition problem.
2. Problem Formulation
Generally, the time-varying LU decomposition problem can be formulated as follows:where denotes a smoothly time-varying matrix to be decomposed, denotes a unit lower triangular matrix, and denotes an upper triangular matrix. Note that and are unknown and time-varying matrices to be obtained.
It is worth noting that not all time-varying matrices have corresponding LU decompositions. In this paper, in order to simplify the problem, we only consider the situation that is a diagonally dominant matrix. It means that (1) is solvable.
3. Continuous-Time Model
In this section, by employing the ZNN method, a CTLUD model is proposed.
According to the previous work of the ZNN method, if we want to find a theoretical solution matrix of a time-varying matrix problem, an appropriate error function is indispensable, where indicates the difference between and actual result matrix . In this paper, is constructed as follows:
Then, is adjusted dynamically by the following ZNN design formula:which indicates that is forced to converge to zero matrix globally and exponentially, where denotes the first-order time derivative of . The design parameter , which is used to adjust the convergence speed. The convergence speed becomes higher as the value of increases.
Note that some elements in and are known. Those known elements do not need to solve. Thus, we construct the following two vectors:where and are composed by all unknown elements of and , respectively. Meanwhile, and denote the elements of and in row and column , respectively. The superscript denotes the matrix transpose operator.
Theorem 1. If is a unit lower triangular matrix, then the following equation holds true:where denotes a column vector obtained by stacking all column vectors of the operational matrix together and denotes an identity matrix. Matrix is defined as follows:where is defined as follows:and denotes a zero matrix. Note that is a vector.
Proof. Let multiply and we havewhereThen, we obtain that , that is, . The proof is completed.
Theorem 2. If is an upper triangular matrix, then the following equation holds true:where matrix is defined as follows:and is defined as
Proof. The proof is similar to that of Theorem 1 and thus omitted.
According to previous statements, we havewhere and denote the first-order time derivatives of and , respectively.
Furthermore, the Kronecker product and vectorization techniques are employed, which are formulated by the following lemma [35, 36].
Lemma 1. If , , , and , then is equivalent towhere symbol denotes the Kronecker product. Note that , , and .
By applying Lemma 1, (4) is equivalent towhich is rewritten asFor simplification, we denoteTherefore, we get
Furthermore, we have the following CTLUD model:where denotes the pseudo-inverse matrix of . Meanwhile, by giving the initial value of , and can be obtained via (20). According to Theorems 1 and 2, and are also obtained. Note that (20) is also a neurodynamics model (which transforms a matrix decomposition problem into a matrix differential equation problem), can obtain the solution in real time, and has the advantages of parallelizability [22, 36]. Those are also the advantages of the proposed CTLUD model. Besides, Figure 1 shows the block diagram about the solving progress of the proposed CTLUD model for the time-varying LU decomposition problem.
Furthermore, we have the following theorem, which discusses the convergence performance of the proposed CTLUD model.
Theorem 3. With being a smoothly time-varying matrix and being always nonsingular, the elements of converge to zero globally and exponentially.
Proof. With denoting the element of in row and column , where and . According to (3), we have . Its solution is mathematically expressed as . As time , exponentially converges to zero. Then, globally and exponentially converges to zero matrix; that is, the solution of the CTLUD model globally and exponentially converges to the theoretical solution. The proof is thus completed.
The following remark discusses the computational complexity of the proposed CTLUD model.
Remark 1. With using big notation, the time complexity of each step in obtaining the proposed CTLUD model is listed as follows. Note that only the highest order in each step is concerned. The time complexity of obtaining : . The time complexity of obtaining : . The time complexity of matrix multiplication operation: . The time complexity of matrix pseudo-inverse operation: .Thus, in the case that we use the traditional serial numerical algorithm, the total time complexity of the proposed CTLUD model is . It is worth pointing out that the proposed CTLUD model is a neurodynamics model and can be implemented by parallel distributed processing. In the case that we use parallel distributed processing, the time complexities of obtaining and can be reduced to and , respectively. Besides, the capability that the ZNN method can find the matrix pseudo-inverse in parallel, has been revealed in many works [37, 38]. The time complexities of matrix multiplication operation and matrix pseudo-inverse operation can be reduced to by sacrificing the space complexity .
4. Discrete-Time Models
In this section, for the convenience of digital hardware realization [34, 39–41], three discrete-time LU decomposition models are proposed, discussed, and investigated. Note that the three discrete-time models are obtained by applying three corresponding discretization formulas.
4.1. Discretization Formulas
The Euler formula , which is also a one-step-ahead discretization formula, is presented as follows:which contains values of two time instants, where and denotes the sampling gap. Besides, denotes the first-order truncation error.
A 4-instant ZeaD formula , which contains values of four time instants, is presented as follows:where denotes the second-order truncation error.
The last discretization formula employed in this paper is an 8-instant ZeaD formula  and presented as follows:where denotes the fourth-order truncation error.
4.2. Corresponding Models
According to the three discretization formulas presented in the last subsection, the following three equations are obtained:where denotes a vector with entries of order and . Based on the above equations, three corresponding discrete-time models are obtained as follows:where “” denotes the computation assignment operator from its right-hand side to its left-hand side. In this paper, (26)–(28) are termed Euler discrete-time LU decomposition (EDTLUD) model, 4-instant ZeaD LU decomposition (4IZLUD) model, and 8-instant ZeaD LU decomposition (8IZLUD) model, respectively. Besides, we have the following theorem.
Theorem 4. With denoting the sampling gap, the EDTLUD, 4IZLUD, and 8IZLUD models are 0-stable, consistent, and convergent, and their truncation error orders are , , and , respectively.
Proof. The EDTLUD model is chosen as an example. The characteristic polynomial of the EDTLUD model is expressed as follows:Evidently, (29) has only one root on unit circle, that is, . According to Lemma 2 in the appendix, the EDTLUD model is 0-stable. Moreover, based on (24), the EDTLUD model has a truncation error of . Therefore, we come to the conclusion that the EDTLUD model is consistent and convergent according to Lemmas 3–5 in the appendix. As for the 4IZLUD model and the 8IZLUD model, the proofs are similar to that of above, and they are thus omitted here. The proof is completed.
Algorithm 1 shows the pseudocode describing the various steps of the proposed 4IZLUD model. As for the EDLUD model and the 8IDLUD model, their operation processes are very similar to that of the 4IDLUD model and are thus omitted. The following remark shows the analyses about the computational complexities of the proposed discrete-time models.
Remark 2. With using big notation, the time complexity of each step in obtaining the solution of the time-varying LU decomposition is listed as follows. Note that only the highest order in each step is concerned. The time complexity of obtaining : . The time complexity of obtaining : . The time complexity of obtaining : . The time complexity of obtaining : . The time complexity of obtaining : .Thus, in the case that we use the traditional serial numerical algorithm, the total time complexity of the proposed discrete-time models is . Similar to the statements in Remark 1, the total time complexity of the proposed discrete-time models can also be reduced to by sacrificing the space complexity .
5. Computer Simulations and Results of CTLUD Model
In this section, for verifying the effectiveness of the proposed CTLUD model, three time-varying matrices are presented. Note that the three matrices have different dimensions, and the initial state of the CTLUD model can be randomly set. In this paper, we randomly use some integers to be the initial state of the CTLUD model in all examples. Besides, the corresponding experiment results are shown.
5.1. Example 1
Firstly, in order to test the performance of the proposed CTLUD model on solving simple LU decomposition problem, the following 2-dimensional time-varying matrix is considered:
In this example, is set as 3 and computation time is limited to s, where denotes the final computation time. In addition, the initial state of is set as
In order to perform the LU decomposition of , the proposed CTLUD model is employed. The corresponding experiment results are shown in Figure 2. As seen from Figure 2(a), the trajectory of is shown, where denotes the Frobenius-norm of . Evidently, converges to 0 in a short time. At about 2 s, is already near 0. Therefore, a conclusion is obtained that the 2-dimensional time-varying LU decomposition problem (i.e., Example 1) is solved effectively by the proposed CTLUD model.
What is more, Figure 2(b) shows the trajectories of , , , and , where denotes the element of in row and column . It can be evidently seen that all elements of converge to 0 in a short time no matter or .
In Figures 2(c) and 2(d), we show the element trajectories of and , where and denote the corresponding theoretical values of and , respectively. As seen from Figures 2(c) and 2(d), all elements of and track their theoretical values. In other examples, the element trajectories of and are not shown because the results are similar to that of Example 1 except for convergence speed.
In this example, because the design parameter is set as 3, the convergence speed is not high enough seemingly, but the convergence process is shown clearly. The experiment results with bigger are shown in Examples 2 and 3.
5.2. Example 2
Furthermore, in order to verify the validity of the proposed CTLUD model for the common LU decomposition problem, we consider the following 3-dimensional time-varying matrix:
Similar to Example 1, the computation time is limited to s. Different from Example 1, the design parameter is set as 30, which is greater than that of Example 1. In addition, the initial state of is set as
By employing the proposed CTLUD model to solve the 3-dimensional time-varying LU decomposition problem, matrices and are obtained. The corresponding experiment results are obtained and shown in Figure 3. In Figure 3(a), the trajectory of is shown. converges to 0 with a high convergence speed. What is more, the convergence speed is much higher than that of Example 1. At about 0.5 s, is already near 0. Thus, the following conclusion is obtained. The 3-dimensional time-varying LU decomposition problem is solved effectively through the proposed CTLUD model.
In addition, Figure 3(b) shows the trajectories of all elements of . From Figure 3(b), all elements of converge to 0 rapidly. In particular, as for , which is much bigger than initial values of other elements, and other elements converge to 0 near-simultaneously. Therefore, the effectiveness of the proposed CTLUD model is verified by the experiment results.
Therefore, for a 3-dimensional time-varying matrix, the CTLUD model can also find its LU decomposition effectively.
5.3. Example 3
Thirdly, in order to observe the performance of model (20) for solving the high-dimensional LU decomposition problem, a 7-dimensional time-varying diagonally dominant matrix is considered, that is, (30).
In this example, the computation time is still limited to s. Different from the previous two examples, the design parameter . What is more, the initial state of is set as
The corresponding experiment results synthesized by the CTLUD model are shown in Figure 4. As seen from Figure 4(a), although is a 7-dimensional matrix, the has converged to 0 in about 1 s, and the convergence speed is higher than that of Example 1 and lower than that of Example 2 because the , which verifies the statements in Section 3.
What is more, the trajectories of all elements of are shown in Figure 4(b). Evidently, all elements converge to 0 near-simultaneously, although the initial values of some elements are bigger than others. As for the high-dimensional and time-varying LU decomposition problem, the proposed CTLUD model is also effective.
In summary, the LU decomposition problems of three time-varying matrices with different dimensions and different initial states, are solved effectively by the CTLUD model.
6. Numerical Experiments and Results of Discrete-Time Models
In this section, in order to verify the effectiveness of three proposed discrete-time models, we use EDTLUD, 4IZLUD, and 8IZLUD models to solve three time-varying LU decomposition problems in the last section, respectively. Besides, the numerical results verify the statements of Theorem 3.
For Example 1, is the error matrix and the residual error is defined as . In this paper, is termed step length. In all experiments involved in this section, is set as 0.03. The results synthesized by the three proposed discrete-time models with , , and are shown in Figure 5. Specifically, Figures 5(a) and 5(b) show the results with and , respectively. Evidently, with different values of , the residual errors of the three models converge to near 0 but with different precision as time goes by. Compared with EDTLUD and 4IZLUD models, the 8IZLUD model has the highest precision for solving the time-varying LU decomposition problem.
It is worth mentioning that, as the value of decreases by 10 times, the maximal steady-state residual errors of three models reduce by , , and times, respectively, which coincides with Theorem 4.
For Example 2 and Example 3, the numerical experiment results synthesized by three models with , , and are shown in Figures 6 and 7, respectively. From Figures 6 and 7, we can obtain a conclusion similar to the one above; that is, the 8IZLUD model has the highest precision among the three discrete-time models.
In summary, numerical results illustrate that the three discrete-time models are effective for solving the time-varying LU decomposition problem.
In this paper, the time-varying LU decomposition problem has been discussed and investigated. By applying the ZNN method, an effective CTLUD model has been proposed. Then, based on the Euler formula, 4-instant ZeaD formula, and 8-instant ZeaD formula, three discrete-time LU decomposition models have been proposed and discussed by discretizing the proposed CTLUD model. Furthermore, in order to verify the effectiveness of the proposed models, three time-varying matrices with different dimensions have been presented. By using the proposed models to perform the LU decomposition of the presented matrices, the corresponding results have been obtained, shown, and discussed. A conclusion has been obtained that the proposed models are effective for solving the time-varying LU decomposition problem, and the 8IZLUD model has the highest precision among the three discrete-time models. More investigations and experiments about other time-varying matrix decompositions and corresponding applications would be future work directions.
Lemma 2. As for a linear -step method described by , whose 0-stability can be checked by determining the roots of its characteristic polynomial , if every root of satisfies with being unique, the linear -step method has strong 0-stability (i.e., is strongly 0-stable).
Lemma 3. With , if the truncation error for the exact solution is of order , the -step method is generally consistent with order .
Lemma 4. For any time instant s, as , if and only if a linear -step method is 0-stable and consistent, the method is convergent, which is written as mathematically. That is, 0-stability plus consistency means convergence, which is also known as the Dahlquist equivalence theorem.
Lemma 5. A linear -step method is convergent with the order of its truncation error, if the method is 0-stable and consistent.
The experiment data used to support the findings of this study are included within the article.
Yunong Zhang is a co-first author.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This work was aided by the National Natural Science Foundation of China (no. 61976230), the Project Supported by Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme (no. 2018), the Key-Area Research and Development Program of Guangzhou (no. 202007030004), the Research Fund Program of Guangdong Key Laboratory of Modern Control Technology (no. 2017B030314165), the China Postdoctoral Science Foundation (no. 2018M643306), the Fundamental Research Funds for the Central Universities (no. 19lgpy227), and also the Shenzhen Science and Technology Plan Project (no. JCYJ20170818154936083).
R. Wu and X. Xie, “Two-stage column block parallel LU factorization algorithm,” IEEE Access, vol. 8, pp. 2645–2655, 2019.View at: Google Scholar
S. Han and Z. Zhang, “A novel zero-watermark algorithm based on LU decomposition in NSST domain,” in Proceedings of the IEEE International Conference on Signal Processing, pp. 1592–1596, Beijing, China, April 2013.View at: Google Scholar
M. Eljammaly, Y. Hanafy, A. Wahdan, and A. Bayoumi, “Hardware implementation of LU decomposition using dataflow architecture on FPGA,” in Proceedings of the International Conference on Computer Science and Information Technology, pp. 298–302, Amman, Jordan, March 2013.View at: Google Scholar
R. Fang, S. Dong, B. Zhu, Q. Xu, and Y. Song, “Improved loop-current-method power flow algorithm for distribution network based on accelerated parallel LU decomposition,” Power System Technology, vol. 46, no. 6, pp. 2179–2186, 2019.View at: Google Scholar
O. Jane, H. G. Ilk, and E. Elbase, “A secure and robust watermarking algorithm based on the combination of DWT, SVD, and LU decomposition with Arnold’s cat map approach,” in Proceedings of the International Conference on Electrical and Electronics Engineering, pp. 306–310, Bursa, Turkey, November 2013.View at: Google Scholar
R. Kastner, “An LU-spatial decomposition method,” in Proceedings of the Antennas and Propagation Society International Symposium, pp. 864–867, Ann Arbor, MI, USA, June 1993.View at: Google Scholar
T. Yigal and K. Raphael, “A recursive LU decomposition algorithm,” Microwave and Optical Technology Letters, vol. 7, no. 2, pp. 48–52, 1994.View at: Google Scholar
Y. Zhang and C. Yi, Zhang Neural Networks and Neural-Dynamic Method, Nova Science Press, New York, NY, USA, 2011.
S. Kawakami and T. Ohtsuki, “Localization using iterative angle of arrival method sharing snapshots of coherent subarrays,” EURASIP Journal on Advances in Signal Processing, vol. 46, pp. 1–7, 2011.View at: Google Scholar
L. Xiao, B. Liao, S. Li, and K. Chen, “Nonlinear recurrent neural networks for finite-time solution of recurrent neural networks for finite-time solution of general time-varying linear matrix equations,” Neural Networks, vol. 98, pp. 102–113, 2017.View at: Google Scholar
J. Li, Y. Zhang, and M. Mao, “General square-pattern discretization formulas via second-order derivative elimination for zeroing neural network illustrated by future optimization,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 3, pp. 891–901, 2019.View at: Publisher Site | Google Scholar
R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, UK, 1991.
Y. Zhang, L. Ming, H. Huang, J. Chen, and Z. Li, “Time-varying Schur decomposition via Zhang neural dynamics,” Neurocomputing, vol. 419, pp. 251–258, 2020.View at: Google Scholar
Y. Zhang, X. Liu, Y. Ling, M. Yang, and H. Huang, “Continuous and discrete Zeroing dynamics models using JMP function array and design formula for solving time-varying Sylvester-transpose matrix inequality,” Numerical Algorithms, vol. 86, no. 4, pp. 1591–1614, 2021.View at: Publisher Site | Google Scholar
Y. Zhang, M. Zhu, C. Hu, J. Li, and M. Yang, “Euler-precision general form of Zhang et al. discretization (ZeaD) formulas, derivation, and numerical experiments,” in Proceedings of the Chinese Control and Decision Conference, pp. 6262–6267, Shenyang, China, June 2018.View at: Google Scholar
E. Suli and D. F. Mayers, An Introduction to Numerical Analysis, Cambridge University Press, Cambridge, UK, 2003.
D. F. Griffiths and D. J. Higham, Numerical Methods for Ordinary Differential Equations: Initial Value Problems, Springer Press, London, UK, 2010.