#### Abstract

A principal singular vector based on multiple Toeplitz matrices is proposed to solve the accuracy problem of direction-of-arrival (DOA) estimation for coherent signals. First, the data matrix received by uniform linear array (ULA) is transformed into a Toeplitz matrix. An equivalent covariance matrix is obtained by square weighted summation method using the Toeplitz matrix. Then, a polynomial containing DOA information is constructed because the signal space and the steering matrix have the same column space; the Toeplitz matrix is built using polynomial coefficients. The problem is transformed into solving linear equations by establishing the relationship between the Toeplitz matrix and the signal subspace. Furthermore, the weighted least square method is used to obtain multiple candidates for linear equations. Finally, the maximum likelihood (ML) rule is used to select source signal candidates from multiple candidates. In comparison with currently known algorithm, the proposed algorithm has the characteristics of high estimation accuracy, low-complexity, and strong anti-interference ability and resolution. Even when the signal-to-noise ratio (*SNR*) is low, the snapshot number is small, and multiple signals exist; this method can still provide good estimation performance and resolution, which is more than 90% in most cases. Simulation experiments verify the superiority of the algorithm.

#### 1. Introduction

Array signal processing is a crucial research issue in signal processing and has been extensively used in radar [1, 2], sensor [3, 4], remote sensing [5–7], target detection [8, 9], and wireless communication [10–12].

As one of the most important research contents in array signal processing, direction-of-arrival (DOA) estimation has been the focus of scholars for decades. Scholars have put forward many excellent algorithms. Among them, DOA algorithm based on subspace is the most representative method in DOA estimation, including multiple signal classification (MUSIC) [13], estimation of signal parameters via rotational invariance techniques (ESPRIT) [14], and its variants [15–20]. MUSIC replaces data matrix by constructing sampling covariance matrix and then obtains DOA by spectral peak search, which has high signal source detection resolution. However, due to coherent signals, the performance of the algorithm degrades and DOA cannot be accurately estimated. ESPRIT algorithm uses the rotation invariance between signal subspaces caused by sensor arrays with translational invariance structure; it is used to obtain high-resolution DOA information. Similar to the MUSIC algorithm, the detection accuracy of DOA decreases seriously in the face of coherent signal detection.

Many techniques and algorithms for processing coherent signals, such as forward-only spatial smoothing (FOSS) [21] and forward/backward spatial smoothing (FBSS) [22], have been proposed by scholars to process coherent signals. These methods divide the total array into several subarrays and then use the average value of the subarray covariance matrix to solve the coherent signal direction finding. However, the disadvantage of these methods is that they reduce the array aperture, resulting in reduced resolution of closely spaced arrivals. In addition, signal subspace fitting (SSF) [23] and maximum likelihood (ML) [24–26] can solve the coherent signal by reducing the multidimensional problem to the 1D problem, without the need for feature decomposition in the process; it is insensitive to the coherence between signals. However, the number of signals greatly influences the estimation accuracy of the algorithm, and more signals indicate greater influence.

Recently, DOA algorithm based on polynomial solution, such as the method of direction estimation (MODE) [27, 28], the enhanced principal-singular-vector utilization for modal analysis (EPUMA) [29], and its variants [30–33], has received remarkable attention from scholars. MODE is similar to ML, but it can process coherent signals without complex computation and has no convergence problems. EPUMA is a low-complexity algorithm; it generates () DOA candidates for sources and then selects to obtain coherent signal DOA information. Scholars have obtained good results by using these algorithms to solve coherent signals. However, in the low *SNR* and small snapshot number, they cannot accurately estimate, because the algorithm cannot accurately obtain sufficient signal information. If the source signal is highly correlated, then this situation will be exacerbated.

In addition, scholars have also proposed other excellent DOA algorithms [34–40]. In [34], a direction-of-arrival estimation algorithm based on low-rank reconstruction of the Toeplitz covariance matrix is proposed by scholars. In order to fully utilize the underlying received information in the presence of missing elements in the difference coarray, interpolation is performed and a dual variable rank minimization problem is formulated. The scholars recast the problem as a multiconvex form and developed an alternative optimization mechanism to solve the problem through cyclic iterations. In [35], a modified scheme based on forward and backward partial Toeplitz matrices reconstruction named as FB-PTMR is proposed. The scholars exploited half rows of the sample covariance matrix (SCM) to reconstruct the data matrix to overcome the performance deterioration of ESPRIT-like algorithm. In [36], the scholars used the existing decoherence algorithm, where the sample covariance matrix of each row is formed into a full-rank Toeplitz matrix to achieve decoherence, and then a new cost function to obtain DOA via a 1-D search. The advantage of the algorithm is that it does not require to know the source number information. In [37], aiming at the fact that conventional algorithms based on the convex relaxation is computationally expensive, the scholars proposed a nonconvex accelerated structured alternating projection-based direction-of-arrival estimation approach without solving semidefinite programming. In [38], the scholars proposed a new real-valued transformation for DOA estimation with arbitrary linear arrays by exploiting the virtual steering of linear arrays, which achieve a better performance in terms of both estimation accuracy and computational complexity. In [39], a DOA algorithm based on correlation matrix rearrangement is proposed. The algorithm can effectively deal with coherent signals in both 1D and 2D. In [40], an algorithm called MTOEP is proposed. MTOEP uses Toeplitz matrix and the observation data of each sensor to calculate a set of correlation matrices and then sums the square weighted of these correlation matrices to form the full-rank equivalent data covariance matrix. However, when multiple related signals and independent signals coexist, the error of this algorithm increases; a higher degree of correlation that exists between signals indicates greater error.

We propose a principal singular vector algorithm based on multi-Toeplitz matrix to eliminate the weakness of the existing algorithm. The proposed algorithm constructs a new covariance matrix and polynomial containing signal information by using the characteristic matrix, where the received data moments and signal space have the same column space as the steering matrix. The estimation problem is transformed into a linear equation problem, and the proposed algorithm has a better resolution effect. This algorithm combines the advantages of Toeplitz matrix and polynomial solution to obtain a high precision estimation algorithm without noise processing and auxiliary matrix. Subsequent experiments prove that the proposed algorithm can process multiple signals with low *SNR*, or the snapshot number is small and still has good performance.

This paper is organized as follows: we introduce the proposed algorithm in Section 2. In Section 3, we perform a series of simulation experiments. Finally, we summarize the study.

##### 1.1. Notations

Superscripts , , , , and represent the matrix pseudoinverse, matrix transpose, matrix conjugate transpose, matrix inversion, conjugate, and traces of matrix, respectively. The operator , , , and indicate diagonalization, expectation, vectorization, and Kronecker product, respectively. and denote identity matrix and zero matrix, respectively.

#### 2. Proposed Algorithm

##### 2.1. Signal Model

The uniform linear array (ULA) consists of 2 M +1 isotropic sensors, each spaced apart, assuming that narrow band signals impinge on the array in the distance. The received signal is expressed as follows: where is the steering matrix, denotes the source signal vector, is Gaussian white noise vector with zero mean and variance , and snapshot number is . The incident angle of the th signal is , and the steering vector due to the th source is expressed as follows: where is the carrier wavelength.

##### 2.2. Equivalent Covariance Matrix Is Constructed Based on Toeplitz Matrix of Data Matrix

We use data matrix to construct Toeplitz matrix, whose expression is as follows: where are the Toeplitz matrix constructed by the and the noise vector , respectively. Therefore, we can obtain the correlation matrix between and the th isotropic sensor output , as follows: where is an matrix whose elements are zero, except the th diagonal, which is unity element. We fully use information to avoid the decrease in the estimation accuracy caused by noise suppression and array output covariance matrix. The advantage of this approach is that the noise term is diagonalized to improve the estimation accuracy without the need for denoising. Then, we have the following:

According to formula (6), an equivalent covariance matrix can be obtained. In addition, to further improve performance, FBSS technology was used to further optimize the obtained equivalent , which is expressed as follows: where is a matrix with antidiagonal, and the other element is zero. Then, singular value decomposition of is obtained as follows: where , , , , and represent the estimated values of , , , , and ; contains the signal eigenvalues, and contains the noise eigenvalues.

##### 2.3. Toeplitz Matrix Containing DOA Information Is Constructed

and have the same column space, according to linear prediction (LP) theory; thus, each column of is a sum of sinusoids. From [41], we obtain the following: where , and represents the LP coefficients. According to LP coefficients, Toeplitz matrix consists of polynomial formula (9) coefficients containing signal information, as follows:

##### 2.4. Multiple DOA Candidates Were Obtained by Solving Linear Equations

The matrix form of formula (9) is as follows: where , , and is the th element in .

Let and become estimates of and , and then we have the following: formula (13) cannot be solved directly due to noise. Therefore, the relationship between Toeplitz matrix and formula (13) should be established. In the case of noise, we find that

By combining formulas (11) and (14), the following can be obtained:

##### 2.5. Weighted Least Square Method Used to Solve the Linear Equation

The weighted least square (WLS) method [42] is used to obtain a more accurate estimate, and the cost function is set as follows: where , and . Then, the solution of formula (16) is as follows: where we replace by . However, we cannot obtain the value of directly, because is unavailable. Thus, we obtain an estimate of by combining formulas (14) and (16), as follows: where is replaced by , and

Then, by plugging formula (18) into formula (17), we have the following:

##### 2.6. Maximum Likelihood Rule Used to Obtain DOA

Multiple are iteratively obtained, and then the maximum likelihood rule is used

to obtain the exact DOA. First, the initial value of is obtained by calculation, which can be obtained from formula (13), as follows: and then formula (21) is substituted into formula (18) to obtain new . The new is substituted into formula (20) to obtain . After several iterations, we obtain all the DOA candidates as follows:

Finally, we select estimates from all DOA candidates by maximum likelihood rule, as follows: where contains the desired DOA information. The proposed algorithm is summarized in Table 1.

##### 2.7. Complexity Analysis

The main steps of the proposed algorithm are as follows: (1)Calculation of and its EVD(2)Calculation of (3)Calculation of

The complexity of the main steps of the proposed algorithm is summarized in Table 2.

The sample covariance matrix and EVD complexity of the proposed algorithm are and , respectively. The computational complexity is mainly caused by , which is about . The complexity of calculating is . Finally, the complexity of DOA is obtained by maximum likelihood rule, that is, . Thus, the complexity of the proposed scheme is: where is the number of iterations, and . Due to and isotropic sensors , the complexity is reduced to .

#### 3. Results and Discussion

In this section, we present a series of experiments to verify the superior performance of the proposed algorithm and consider ESPRIT [14], MODE [27], MODEX [31], EPUMA [29], FB-PTMR [35], and MTOEP [40] for comparison. The Cramér–Rao Bound (CRB) is used as a measure of performance in the experiment. In addition, FBSS [21] and FOSS [22] technologies are used to improve the ability of ESPRIT to process coherent signals. During the experiment, we assume that all signals are narrowband signals, and the ULA composed of sensors is used to receive signals. In order to demonstrate the advantage of the proposed algorithm in dealing with coherent signals, in the following experiments, coherence coefficient is 1. Furthermore, 3000 Monte Carlo experiments are performed for each experiment to obtain more accurate experimental data. The root mean square error (RMSE) of DOA is as follows: where is the experimental value and is the true value.

In the first experiment, we study the effect of different *SNR* on the RMSE and resolution of the algorithm. Three signals with DOAs being [–5°, 0°, 25°]are consider (*K* =3), where the first two signals are coherent and uncorrelated with the third one. We can obtain RMSE from (25), and the snapshot number is . As shown in Figure 1, when dB, the RMSE of all algorithms are far from the CRB, FB-PTMR, and MTOEP, and the proposed algorithms are slightly better than the other algorithms. When dB and , the RMSE of all algorithms is reduced, as well as the FB-PTMR, MTOEP, and the proposed converge to the CRB when dB. Meanwhile, due to the lack of array aperture, although ESPRIT optimized by FOSS and FBSS can process coherent signals, the RMSE of the optimized ESPRIT does not completely converge to the CRB until dB. The performance is the best when using EPUMA and MODEX on dB, MODE always has a large distance from the CRB, and its ability to process coherent signals is poor.

**(a)**

**(b)**

To further verify the performance of the proposed algorithm, an uncorrelated signal () is added. Figure 2 shows that, similar to , when dB, the RMSE of all algorithms is large and far from the CRB. The RMSE of the proposed algorithm is smaller than that of other algorithms. When dB and dB, the proposed algorithm and EPUMA converge to the CRB, respectively. FB-PTMR and MTOEP still have a gap with the CRB even when dB. Compared with , when , only the proposed algorithm is unaffected by the number of signals, whereas the other algorithms are affected by the number of signals, among which MODEX is the most affected.

**(a)**

**(b)**

The influence of *SNR* on algorithm resolution is shown in Figures 3 and 4. When , *SNR*<0 dB, the resolution of FB-PTMR, MTOEP, and the proposed algorithm is lower than 80%, whereas other algorithms are lower than 50%. With the increase in *SNR*, the resolution of all algorithms increases rapidly. Except for MODE, when dB, the performance reached or exceeded 90%. When , dB, the resolution of the proposed algorithm is much higher than that of the other algorithms but lower than 80%. When dB, only the resolution of EPUMA, MTOEP, FB-PTMR, and the proposed algorithm reaches or even exceeds 90%, whereas that of the other algorithms is lower than 90%.

**(a)**

**(b)**

**(a)**

**(b)**

This group of experiments verifies that the proposed algorithm has good anti-interference ability and can still effectively process coherent signals when facing multiple signals and different *SNR*, with small error and high resolution, which is unavailable in other algorithms. Under certain conditions, the resolution of the proposed algorithm even reaches 95%, whereas that of other algorithms is greatly affected by the number of signals. In addition, the error increases evidently, and the resolution decreases when processing multiple signals. Furthermore, this group of experiments show the advantage of the proposed algorithm in processing coherent signals at low *SNR*, and the next experiments will further study the ability of the proposed algorithm to handle coherent signals in different situations when dB.

In the second experiment, we study the effect of different snapshot numbers on the RMSE and resolution of the algorithm. The experimental conditions are the same as the first experiment. Figure 5 shows that the performance of the proposed algorithm is better than that of the other algorithms. When , with the increase in snapshot number, the proposed algorithm converges rapidly, and RMSE decreases. Although the RMSE of MODEX, FB-PTMR, MTOEP, and EPUMA is also reduced, it is not as good as the proposed algorithm. However, FOSS, FBSS, and MODE almost remain unchanged, and their errors are much higher than those of the other algorithms, even if the snapshot number is 1000. When , the proposed algorithm still has better processing ability and is unaffected by the number of signals. EPUMA is less affected by the number of affected signals, and RMSE is basically unchanged, but not as good as the proposed algorithm. However, the other algorithms are greatly affected by the number of signals, and the RMSE increases.

**(a)**

**(b)**

Figure 6 shows that when , the resolution of EPUMA and MTOEP can be close to or reaches 80%, whereas when , the resolution of MTOEP decreases seriously and is lower than 50%. Other algorithms have low resolution in any case, but the proposed algorithm can still have a resolution higher than 90% under any condition.

**(a)**

**(b)**

Therefore, compared with other algorithms, the proposed algorithm can effectively use snapshot to detect signal information, and the estimation accuracy is high. It can process multiple signals with a high resolution of more than 90%, whereas other algorithms are inferior to the proposed algorithm with a low resolution.

In the third experiment, we study the effect of different angular separation on the RMSE and resolution of the algorithm. The parameters remained the same as in the first experiment. In Figure 7, when and separation is 9, the MTOEP and the proposed algorithm reach the optimal state. In addition, the other algorithms are not at their best even when the separation is 10. When , only EPUMA and the proposed algorithm are less affected by the number of signals, with RMSE decreasing significantly as separation increases, whereas the RMSE of the other algorithms decreases but not as much as EPUMA and the proposed algorithm.

**(a)**

**(b)**

As in the previous experiment, we also tested the angular separation impact on algorithm resolution. As shown in Figure 8, when , the resolution of EPUMA and the proposed algorithm reaches or even exceeds 90%. The resolution of MTOEP, FB-PTMR, and MODEX also exceeds 80%, and that of the remaining algorithms does not exceed 60%. When , the resolution of all algorithms decreases seriously. The performance of EPUMA and the proposed algorithm is reduced from 90% to about 80%, whereas that of other algorithms is lower than or equal to 30%.

**(a)**

**(b)**

This set of experiments demonstrates that the proposed algorithm is relatively affected when dealing with angular separation variations of multiple coherent signals (), but the resolution is still higher than that of the other algorithms under the same conditions; the resolution of the algorithm reaches 80%. When the number of signals is small (), the resolution of the proposed algorithm can still exceed 90%. The performance of other algorithms is far inferior to that of the proposed algorithm.

Finally, because FB-PTMR, FBSS, and FOSS are all based on ESPRIT algorithm, MODEX complexity is extremely huge, so we compare the complexity of ESPRIT, MODE, MTOEP, EPUMA, and the proposed algorithm. We assume that three signals are available, is 80, and we vary from 10 to 45. Through calculation, we obtain that the complexity of ESPRIT, MODE, MTOEP, and EPUMA is , , , and , respectively. Figure 9 shows that ESPRIT has simple computational complexity, whereas the standard MODE is the most computationally intensive method among the six competitors. The computational complexity of the proposed algorithm and EPUMA is similar.

#### 4. Conclusions

An efficient algorithm without auxiliary matrix and noise preprocessing is proposed by combining the advantages of polynomial solving DOA signal and Toeplitz matrix reconstruction. In the face of multiple signals, the performance of other reference algorithms has been greatly affected, whereas the proposed algorithm still has high estimation accuracy. Simulation experiments show that the proposed algorithm has the advantage of processing multiple signals containing coherent signals, and it is less affected by *SNR* and snapshot number, which is an advantage that other reference algorithms do not have. Under certain conditions, the resolution can reach 90% and sometimes exceed 95%. Although the proposed algorithm has many advantages, when angular separation changes and with multiple signals (), the proposed algorithm is affected to some extent, and the resolution is only 80%. However, when , the resolution immediately increases to 95%. In addition, the proposed algorithm has low complexity. This paper provides a signal location method with high estimation accuracy, strong anti-interference ability, and effective processing of multiple signals.

#### Data Availability

Data are only available upon request due to restrictions regarding, e.g., privacy and ethics. The data presented in this study are available from the corresponding author upon request. The data are not publicly available due to their relation to other ongoing research.

#### Conflicts of Interest

The authors declare no conflict of interest.

#### Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 61763018), the Central Guided Local Science and Technology Funding Project of the Science and Technology Department of Jiangxi Province (Cross-regional Cooperation, 20221ZDH04052), the 03 Special Project and 5G Program of the Science and Technology Department of Jiangxi Province (No. 20193ABC03A058), the China Scholarship Council (CSC, No. 201708360150), the Key Foundation of Education Committee of Jiangxi (Nos. GJJ170493 and GJJ190451), the Program of Qingjiang Excellent Young Talents in Jiangxi University of Science and Technology (JXUSTQJBJ2019004), and the Cultivation Project of the State Key Laboratory of Green Development and High-Value Utilization of Ionic Rare-Earth Resources in Jiangxi Province (20194AFD44003), the Key Research and Development Plan of Ganzhou (industrial field), and the Science and Technology Innovation Talent Project of Ganzhou.