#### Abstract

A novel direction-of-arrival (DOA) estimation method is proposed to cope with the scenario where a number of uncorrelated and coherent narrowband sources simultaneously impinge on the far-field of a uniform linear array (ULA). In the proposed method, the DOAs of uncorrelated sources are firstly estimated by utilizing the property of the moduli of eigenvalues of the DOA matrix. Afterwards, the contributions of uncorrelated sources and the interference of noise are eliminated completely by exploiting the improved spatial differencing technique and only the coherent components remain in the spatial differencing matrix. Finally, the remaining coherent sources can be resolved by performing the improved spatial smoothing scheme on the spatial differencing matrix. The presented method can resolve more number of sources than that of the array elements and distinguish the uncorrelated and coherent sources that come from the same direction as well as improving the estimation performance. Simulation results demonstrate the effectiveness and efficiency of the proposed method.

#### 1. Introduction

Direction-of-arrival (DOA) estimation is a major research issue in array signal processing including radar, guidance systems, sonar, seismic exploration, and electronic surveillance [1]. Several high-resolution algorithms, such as multiple signal classification (MUSIC) [2] and estimation of signal parameter via rotation invariance techniques (ESPRIT) [3], have been proposed to resolve the far-field uncorrelated sources. However, highly correlated or coherent source signals are common in multipath propagation environments due to the reflection and refraction of source signals in practical. Based on such scenario, the coherent sources facilitate the rank loss of the covariance matrix, which could result in the failure of the conventional high-resolution estimation algorithms. That is, those high-resolution algorithms may fail in localizing when uncorrelated and coherent source signals coexist.

To solve the aforementioned coherency problem, various effective techniques have been proposed. Some preprocessing techniques referred to as spatial smoothing (SS) have been developed. The SS method divides the total array elements into a few overlapping subarrays and then averages the subarray output covariance matrices to form the spatially smoothed covariance matrix to decorrelate the coherence between the incoming sources [4, 5]. However, the SS method generally reduces the array aperture. Besides, the number of signals the SS method resolved cannot exceed the number of array sensors. In [6], a JADE approach is presented to estimating the angle-of-arrival and delays of the multipath source signals, which can estimate more parameters than the number of antennas. A DOA estimation technique [7] is proposed to eliminate the possible false DOAs of uncorrelated signals. Unfortunately, this method shows an unsatisfactory estimation performance. The approach based on higher-order cumulants (HOC), such as [8, 9], can estimate the DOAs of coherent source signals. But this method generally requires large number of snapshots and suffers from burdensome computation. The algorithm proposed in [10] decorrelates the coherent sources by reconstructing a Toeplitz matrix and achieves good performance. However, the main disadvantage is that the number of resolved sources is restricted within the number of reduced array sensors no matter whether the sources are coherent or not. In [11], a deflation approach is introduced, but the number of resolvable coherent sources is less than half of the number of array elements. Recently, a relevance vector machine algorithm [12], based on spatial filtering, is proposed to estimate DOAs of coherent incoming signals. However, it can only deal with coherent sources and the number of sources resolved by this method is less than that of array elements. In order to deal with more sources, a non-Toeplitz matrix is constructed by exploiting the symmetric configuration of uniform linear array (ULA) [13]. However, the computational load is too high for practical application when all of the constructed matrices are utilized, whereas the performance degrades if just one constructed matrix is used. In [14], the contributions of uncorrelated and partially correlated source signals are removed by exploiting the oblique projection (OP algorithm). However, it needs to estimate the DOAs of uncorrelated and coherent sources in sequence. Besides, the estimation performance is unsatisfactory. The approach with fewer sensors is presented in [15]. In this approach, the DOA matrix is directly constructed by performing multiple eigenvalue decomposition (EVD) on the covariance matrix, which could result in a high probability of failure; moreover, the computational complexity is not very attractive. The approach introduced in [16] exploits the property of the moduli of eigenvalues to distinguish uncorrelated sources from coherent sources. However, this approach cannot completely eliminate the cross-term effects and the computational complexity is not very attractive, either. The differencing method is introduced in [17–19]. Based on covariance differencing and iterative spatial smoothing, the method introduced in [17] requires information about the covariance matrix of uncorrelated source signals, which may be difficult in realization. The number of sources resolved by the technique in [18] is less than that of array elements. The method in [19] can resolve more source signals. However, the difference smoothing matrix will lead to rank deficient once the coherent group contains an odd number of source signals. Therefore, it needs extra processing to recover the rank. Furthermore, the performance of the method is just verified by simulation results without any theoretical analysis.

In this paper, a high-resolution DOA estimation method is proposed when uncorrelated and coherent source signals are together. Firstly, the uncorrelated sources are distinguished from coherent sources by using the property of the moduli of eigenvalues. Then based on the improved spatial differencing method, the contributions of uncorrelated source signals and the interference of noise are removed such that only coherent source signals are reserved in the constructed differencing matrix. Finally, by performing the improved SS method on the differencing matrix, whose rank is equal to the number of coherent sources, the coherent sources can be achieved.

The rest of the paper is organized as follows. The signal model is briefly introduced in Section 2. The algorithm of DOA estimation is explained in detail in Section 3. In Section 4, simulation results are presented to validate the effectiveness of the proposed algorithm. Finally, some concluding remarks for the proposed algorithm are provided in Section 5.

Throughout this paper, the following notations are used. , , and denote the transpose, conjugate transpose, and pseudo inverse of a matrix, respectively. The notation stands for the moduli of a complex scalar, while is the expectation operator. Moreover, the notation represents the block diagonal matrix with diagonal entries , . The notation denotes submatrix containing the elements from th to th rows and th to th columns of matrix , respectively.

#### 2. Signal Model

Consider narrowband far-field signals with impinging on a ULA with equispaced sensors, where the distance between adjacent sensors is equal to half the wavelength. Let the first sensor be the reference, and then the steering vector can be given:where denotes the carrier wavelength of the source signal. Without loss of generality, assume that the first source signals are uncorrelated, and the source signal that comes from direction corresponds to the propagation of the far-field source signal with power for . The remaining are groups of coherent source signals, which come from statistically independent far-field source signals with power , (), and with multipath sources for each source. Furthermore, in the th coherent group, the source that comes from direction corresponds to the th multipath propagation of source , for and . Assume that the coherent source signals in different groups are uncorrelated with each other and the uncorrelated source signals. The received data vector is given bywhere is the steering vector. is the complex fading coefficient of the th multipath propagation corresponding to the th source signal with , . Also , , with , . Moreover , . is the noise vector with the power of each entry being equal to ; . By assumption, the entries of and are zero-mean wide-sense stationary random processes and are uncorrelated to each other.

From (2), the array covariance matrix can be expressed aswhere is block diagonal. is the covariance matrix of , and is the covariance matrix of . denotes the identity matrix.

#### 3. DOA Estimation of Proposed Method

In this section, the DOA estimation will be carried out by using the proposed method. The processes of the estimation of uncorrelated and coherent sources are described in detail in Sections 3.1 and 3.2, respectively.

##### 3.1. DOA Estimation of the Uncorrelated Sources

For the DOA estimation of the uncorrelated source signals, the EVD of** R** can be expressed:where , , , and . The columns of span the signal subspace corresponding to the larger eigenvalues and the noise subspace is constructed by the eigenvectors corresponding to the smaller eigenvalues. Furthermore, the is also spanned by . Thereforewhere is a full-rank matrix. The can be divided into two partially overlapped subarrays of size . Then the two output submatrix can be expressed aswhere .

Based on the above definition (6), a new matrix can be constructed as follows:where represents the th uncorrelated source and represents the th source in the th coherent group.

According to mathematics knowledge, one can prove that possesses the following important property [16]:

Equation (9) implies that, by performing EVD of , the moduli values of the eigenvalues corresponding to the uncorrelated sources are all equal to 1. Meanwhile, the moduli values of the eigenvalues of the remaining coherent sources will not possess the characteristic of uncorrelated sources, whose moduli values are all less than 1. Therefore, we can choose a threshold in practice to estimate the number of uncorrelated sources:

Substitute the moduli of the eigenvalues , which is in descending order, into (10). If the is the first to satisfy , then is the estimated number of the uncorrelated sources. That is, unlike the conventional approach as in [20], extra process will be avoided in the proposed algorithm to detect the number of sources. Suppose that are the eigenvalues corresponding to the uncorrelated sources, and then the DOAs of uncorrelated sources can be obtained by computing

##### 3.2. DOA Estimation of the Coherent Sources

In this subsection, the improved spatial differencing technique is performed to resolve the DOAs of the coherent sources. From (4), the can be achieved by the mean of the eigenvalues of , which is expressed as follows:

According to (3) and (12), a new matrix can be further obtained:

Due to the influence of the finite samples, still includes the noise residual matrix in practical application. That is, the noise part cannot be eliminated completely in this step. Fortunately, (3) shows that the covariance matrix can be expressed as the sum of a Toeplitz matrix consisting of information on uncorrelated signals, a non-Toeplitz matrix containing information on coherent signals, and the noise covariance matrix. Since any Toeplitz matrix satisfies the following property:where denotes the exchange matrix with ones on its antidiagonal and zeros elsewhere, therefore, this property can be used to cancel out the Toeplitz component and eliminate the residual noise part thoroughly, and then a spatial differencing matrix is defined as follows:

Equation (15) clearly shows that only the information on coherent sources remains in the spatial differencing matrix . Then the EVD of the is achieved as follows:where with and corresponding to the positive and negative eigenvalues, respectively. Furthermore, the columns of are the eigenvectors corresponding to the aforementioned nonzero eigenvalues. Then, a new matrix can be formed by taking the absolute values of the eigenvalues in :

Afterwards, the SS technique is performed on the new matrix . Here, it is assumed that the number of subarrays is , and the size of the subarray is . Thus, , and the th subarray covariance matrix () is expressed as follows:

Therefore, the smoothed matrix can be obtained:

Finally, by applying the high-resolution DOA techniques to , the coherent sources can be obtained as long as and .

Till now, under the coexistence of both uncorrelated and coherent source signals, the proposed method with the finite sampling data can be implemented as follows.

*Step 1. *Collect data and estimate the covariance matrix by (3).

*Step 2. *Obtain the signal subspace by performing the EVD of the matrix , and calculate and according to (6), respectively.

*Step 3. *Estimate the number of the uncorrelated sources by making full use of as in (10), and obtain the DOAs of uncorrelated sources based on (11).

*Step 4. *Calculate the spatial differencing matrix as in (16), and construct the new matrix as in (17).

*Step 5. *Perform SS technique on to obtain the smoothed matrix by (19).

*Step 6. *Estimate the DOAs of coherent sources by making use of high-resolution DOA methods on .

##### 3.3. Discussion

In this subsection, the advantages of the proposed method are discussed. In the proposed method, the uncorrelated and coherent source signals are estimated separately. That is, when an uncorrelated source signal comes from the same direction as a coherent source signal does, the presented method can still distinguish them. Furthermore, the proposed method is still valid when the maximal number of the incident source signals is greater than that of the array elements, which can be considered as another advantage. From the aforementioned analysis, the proposed method can resolve, at most, uncorrelated source signals, and the maximal number of incident coherent source signals is equal to , where denotes the minimal integer no less than . Thus, to resolve all the incoming source signals, the proposed method requires no less than maximum array elements, while the FBSS in [5] requires . It is noteworthy that the proposed method can largely reduce the required elements compared with the FBSS method.

The proposed method can suppress the effects of the uncorrelated sources and the additive Gaussian noise effectively based on the fact that the uncorrelated sources and the noise can be eliminated completely. Therefore, it can yield better DOA estimation than the compared methods in [5, 14]. To sum up, the proposed method offers three main advantages: resolving more sources, achieving better DOA estimation, and less restricting Gaussian noise fields.

#### 4. Simulation Results

In this section, simulation results are presented to illustrate the validity of the proposed method. The array is an eight-element ULA with half wavelength interspacing. For simplicity, assume that all source signals are of equal power . The SNR and threshold are set to and 0.015, respectively. When using (11) and (19) to estimate the uncorrelated and coherent sources, respectively, the search range is performed over −90° to 90° with the scanning interval 0.1°. All the simulation experiments are based on 200 Monte Carlo trials. Two performance indices, called the root-mean-square error (RMSE) and normalized probability of success (NPS), are defined to evaluate the performance of the proposed method:where is the estimate of for the th Monte Carlo trial and is the number of all the uncorrelated or all the coherent sources:where and denote the times of success and Monte Carlo trial, respectively. Furthermore, a successful experiment is the one satisfying . Where equals 0.5 and 1.5 for estimation of uncorrelated and coherent sources, respectively.

In the first simulation, we consider the scenario in which the number of incident sources goes beyond the number of array sensors. Consider four uncorrelated sources coming from [−45°, −10°, 10°, 25°] and two groups of five coherent sources coming from [−45°, −25°, 0°] and [20°, 40°], respectively, when . Note that one of uncorrelated sources, namely, , has the same DOA with one of the first group of coherent sources. The fading coefficients of the two groups of coherent sources are [1, 0.93, 0.89] and [1, 0.9], respectively. The number of snapshots is 1000, and the input SNR is 10 dB. The spatial spectrums of the uncorrelated and coherent source signals by the proposed method are shown in Figure 1. It can be seen that the sharp peaks are detected at the correct DOAs. Moreover, the uncorrelated source from −45° and the coherent source from −45° both can be detected due to the fact that the DOAs of uncorrelated sources and coherent sources are estimated in two stages. This is consistent with the theoretical analysis aforementioned.

The second simulation considers three uncorrelated source signals from [−45°, 15°, 30°] and two groups of four coherent source signals from [−30°, 11°] and [45°, 60°], respectively, when . The fading coefficients of the two groups of coherent sources are [1, 0.9] and [1, 0.85], respectively. The number of snapshots is set to 1000. The RMSE of the DOAs versus input SNR is shown in Figure 2. Figure 2 illustrates that the RMSE of both uncorrelated and coherent sources of the proposed method is more accurate than that of algorithms in [5] (FBSS) and [14] (OP), especially at low SNR. Figure 3 shows the NPS of the DOAs versus SNR, which illustrates that the performance of the proposed method is better than that of FBSS and OP algorithms as the SNR increases. The reason is that the proposed algorithm utilizes the whole ULA to estimate the uncorrelated source signals, and then the interference of Gaussian noise and the contributions of uncorrelated source signals can be fully eliminated in a subsequent stage through the improved differencing processing. On the contrary, based on the reduced array, the FBSS obtains the DOAs of uncorrelated and coherent sources simultaneously. Besides, no power or information is lost for coherent source signals in the proposed method, which is the opposite in OP algorithm.

The third simulation considers the same scenario as the second one at different number of snapshots. When the SNR is 5 dB, the RMSE of the uncorrelated and coherent source signals against number of snapshots is shown in Figure 4. It can be seen from Figure 4 that the proposed method achieves higher estimation accuracy than FBSS and OP algorithms as the number of snapshots increases. The reason is that (11) (estimating the uncorrelated source signals) and (19) (estimating the coherent source signals) are closer to their true value as the number of snapshots increases. Furthermore, no matter how large the number of the snapshots is, the proposed algorithm always has higher estimation performance than FBSS and OP algorithms.

In addition, the NPS of the uncorrelated and coherent sources against number of snapshots is shown in Figure 5 when the SNR is 10 dB. From Figure 5, we can see that the performance of uncorrelated sources by the proposed algorithm is superior to the compared methods as the number of snapshots is varied from 50 to 250. That is, in small number of snapshots, the superiority of the proposed algorithm is more pronounced. This indicates that the proposed algorithm will be more useful when the low-computational cost and highly real-time data process are required. Meanwhile, Figure 5 also shows that the performance of coherent sources by the proposed algorithm outperforms the OP algorithm when the number of snapshots changes. Moreover, the NPS of the proposed algorithm becomes stabilized as the number of snapshots is over 150, while the FBSS will fail in this scenario.

The last simulation studies not only the scenario where the uncorrelated, partially correlated, and coherent sources coexist, but also the situation in which the total number of incoming signals exceeds that of array sensors. The number of snapshots is 1000. Four uncorrelated sources from [−45°, −10°, 10°, 25°], two partially correlated sources from [40°, 60°], and a group of three coherent sources from [−45°, −30°, 11°] are considered when . The fading coefficients of the coherent sources are [1, 0.95, 0.85], and the correlation coefficient between the two partially correlated sources is selected to be . The DOAs of partially correlated sources are estimated by the method in Section 3.2, and the definition of a successful experiment of the partially correlated sources is consistent with that of the coherent sources. The performance curves of the DOAs versus input SNR are shown in Figures 6 and 7. Both figures indicate that the proposed algorithm can still achieve good estimation performance when the uncorrelated, partially correlated, and coherent sources coexist.

Similar to the above simulation, the SNR is set to be 5 dB, and the number of snapshots is varied from 300 to 800. The performance curves of the DOA estimates against snapshots are shown in Figures 8 and 9. Apparently, as shown in Figures 6 and 7, the same conclusions can be drawn. Figures 8 and 9 indicate that the performance of the DOA estimation by the proposed method will be improved as the number of snapshots increases. In addition, all the performance curves will become stabilized as the number of snapshots increases.

#### 5. Conclusions

In this paper, a novel method for DOA estimation is proposed when both uncorrelated and coherent sources coexist. In the proposed method, the uncorrelated sources are estimated by using the moduli of eigenvalues, and then the coherent sources are resolved by the improved spatial differencing smoothing method. The proposed method can resolve greater number of sources than that of array sensors as well as improving estimation performance especially when the SNR is low and the number of snapshots is small. Simulation results validate the effectiveness of the proposed method. Moreover, the presented method can also be extended to the scenario where uncorrelated, partially correlated, and coherent sources are mixed.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

#### Acknowledgments

This work was supported partially by the National Natural Science Foundation of China (no. 61002010) and the project of the State Key Laboratory of Millimeter Waves (no. K201314).