#### Abstract

Due to the fluctuation of the signal-to-noise ratio (SNR) and the single snapshot case in the MIMO HF sky-wave radar system, the accuracy of the online estimation of the mutual coupling coefficients matrix of the uniform rectangle array (URA) might be degraded by the classical approach, especially in the case of low SNR. In this paper, an Online Particle Mean-Shift Approach (OPMA) is proposed, which is to get a relatively more effective estimation of the mutual coupling coefficients matrix with the low SNR. Firstly, the spatial smoothing technique combined with the MUSIC algorithm of URA is introduced for the DOA estimation of the multiple targets in the case of single snapshot which are taken as coherent sources. Then, based on the idea of the particle filter, the online particles with a moderate computational complexity are used to generate some different estimation results. Finally, the mean-shift algorithm is applied to get a more robust estimate of the equivalent mutual coupling coefficients matrix. The simulation results demonstrate the validity of the proposed approach in terms of the success probability, the statistics of bias, and the variance. The proposed approach is more robust and more accurate than the other two approaches.

#### 1. Introduction

Multiple-input and multiple-output (MIMO) radar systems, characterized by multiple antenna elements at the transmitter and the receiver, have demonstrated the great potential for increased ability of the target detection [1–7]. One important application of MIMO radar system is the HF radar [2–5], including the HF sky-wave radar and the HF ground-wave radar. However, one important challenge in MIMO systems is that the mutual coupling becomes particularly significant as the antenna element spacing is decreased. In many practical problems, the direction of arrival (DOA) is the significant information, while at the same time many classical DOA estimation algorithms suffer from sensitivity to mutual coupling, such as MUSIC algorithm [8]. Hence, the DOA estimation for the multiple narrowband signals has been a classical problem in the array signal processing. High-resolution array-processing algorithms for source localization are known to be sensitive to errors in the model for the sensor-array spatial response. In particular, unknown gain, phase, and mutual coupling as well as errors in the sensor positions can seriously degrade the performances of array-processing algorithms. With few exceptions, high-resolution source localization algorithms require an exact characterization of array, including knowledge of the sensor positions, sensor gain/phase response, mutual coupling, and receiver equipment effects. All such information is inevitably subject to errors [9]. The presence of mutual coupling distorts the phase vectors of radiation sources and the eigenstructure of the covariance matrix [10]. In [11], the effects of the mutual coupling on the direction finding accuracy of a linear array with dipole elements were studied. It was found that a known coupling did not affect the estimation performance much. In the case of an unknown mutual coupling, the performance of most high-resolution direction finding algorithms could be degraded. Therefore, a relatively precise coupling coefficient could well establish the high-resolution DOA.

In recent years, a great effort has been seen in the algorithms of the calibration and the coupling coefficients estimation [12–25]. The paper [12] described a calibration algorithm that estimated the calibration matrix consisting of the unknown gain, phase, and mutual coupling coefficients as well as the sensor positions by using a set of calibration sources in known locations, which was based on a maximum likelihood approach. However, this method requires a set of calibration sources at known locations. Two methods to compensate the unknown mutual coupling were proposed in [13], while, just the same as [12], calibration sources were required in both of them. Unlike previous array calibration methods, literature [14] proposed an algorithm that was able to calibrate the array parameters without the prior knowledge of the array manifold. Literature [15] presented a new array calibration procedure for over-the-horizon (OTH) radar, using disparate sources. The method in [16] used the signals impinging on the array to carry out both the DOA estimation and the array calibration simultaneously. In order to express the coupling coefficients, the array coupling matrix was investigated in [17, 18]. Literature [17] proposed a robust subspace-based DOA estimation algorithm in the presence of mutual coupling for ULA, which was based on the banded symmetric Toeplitz matrix model. An accurate estimate of mutual coupling matrix could be achieved simultaneously for array calibration. The method of moments (MoM) was used in [18] to evaluate the mutual coupling between the elements of a given array. The MoM admittance matrix was then used to eliminate the effects of mutual coupling. Reducing the effect of mutual coupling is also an important method for high-resolution DOA estimation [19–21]. The mutual coupling effect of a compact uniform circular array (UCA) is shown in [19] in order not to affect the favorable characteristics of the FFT-based preprocessing technique but only results in a modulation of the signal component at the receiver with a diagonal matrix. In [20], the author divided the antenna into two or more identical subarrays and discussed what kind of errors mutual coupling introduced to the accuracy of the ESPRIT algorithm. In [21], the author estimated the angles when mutual coupling was significant with dummy elements. It has been shown in the pioneering work in [22] that, by applying a group of auxiliary array elements, the MUSIC algorithm can be adopted directly for DOA estimation for ULA at first. Once preliminary DOAs are estimated, the coupling coefficients can be estimated. Then, given the estimated coupling information, the full antenna array with an enlarged array output vector can be processed to refine the DOA estimation. In [23], the 2D DOA estimation in the presence of mutual coupling was presented by setting the sensors on the boundary of the URA as auxiliary sensors. In [24, 25], the modeling and estimation of mutual coupling in a uniform linear array of dipoles were discussed and a method of the mutual coupling compensation using subspace fitting was presented.

However, in MIMO HF sky-wave radar, the application of DOA estimation is often in the domain of the two-dimensional range-Doppler spectra, the peak of which is taken as the single snapshot for the receive array signal processing. The classical approaches mentioned above might be degraded in the case of the single snapshot and the low SNR. In this paper, an online particle mean-shift approach (OPMA) for the robust antenna mutual coupling coefficients estimation with the single snapshot is proposed. The goal is to find an optimal or a suboptimal estimation for the mutual coupling coefficients matrix in the case of the single snapshot. Due to the fact that the estimation of the unknown mutual coupling coefficients is related to the additive noise, the estimated mutual coupling coefficients matrix is modeled as a posteriori probability density function. Moreover, a tracking process based on the idea of particle filter and the mean-shift algorithm is applied to search the most rapid increase of the probability density function.

This paper is organized as follows. In Section 2, the problem and the system model are briefly formulated. The spatial smoothing technique of the uniform rectangle array is introduced. Moreover, the equivalent equation of the mutual coupling coefficients matrix is derived. An OPMA is utilized in Section 3 to further improve the precision of the mutual coupling coefficients matrix estimation. Simulation results for different scenarios are shown in Section 4 and conclusions will follow in Section 5.

#### 2. Problem Formulation and System Model

Consider a collocated MIMO HF sky-wave radar system with the assumption of the narrowband waveform [2]. The transmit antenna is a uniform linear array. The receive antenna is a uniform rectangle array (URA) which is shown in Figure 1.

Let the transmit antenna array consist of antenna elements. The receive antenna array is located at the plane. Assume there are receive antenna elements along the -axis and receive antenna elements along -axis. The spacing of two adjacent elements in the transmit array is . The center wavelength is . The spacing between adjacent rows of the receive array is and the spacing between adjacent columns of the receive array is , and we have . Assume the matrix is the receive data matrix of the time , the element of which is the receive data of the (, ) antenna element denoted as . Let denote the group delay (range) of the th target (), where is the number of the targets. Also, let denote the direction of departure (DOD) of elevation from the transmit antenna to the target, and and denote the direction of arrival (DOA) of azimuth and elevation from the target to the receive antenna. Assume is the mutual coupling coefficients matrix of the receive antenna array and the mutual coupling of the transmit antenna array is ignored. Let denote the -variate time -dependent vector of the received data matrix [2], and we havewhere is the receive antenna steering matrix and is a vectorization operator; we havewhere is the noise vector, the elements of which are assumed to be stationary, zero-mean Gaussian random processes. And is the waveform of a planewave arriving from the target DOA . may be specified aswhere is a unit-norm waveform transmitted by the th element and is the scatter (target) reflection coefficient. The effect of the multiple waveform transmission is given in with as the complex coefficient introduced by transmission of the th waveform from the th array element to th target in the direction of departure (DOD) .

As shown in [22–25], it is often sufficient to consider that the mutual coupling coefficients between adjacent elements are almost the same and the magnitude of the mutual coupling coefficients decreases with increasing the antenna element spacing. In this paper, we assume the one antenna element is only affected by the coupling of the antenna elements, the spacing of which is within , where is a threshold. Then, we havewhere are submatrices of and are written aswhere is the Toeplitz operator. And is the mutual coupling coefficients between two antenna elements, the spacing of which is , where can be .

In HF OTH radar, the application of DOA estimation or the adaptive beamforming is often in the domain of the two-dimensional range-Doppler spectra, where the “spike” peak of the target spectra is approximately occupied by several spectrum bins [26], one of which has maximum SNR within the target spectrum region. Therefore, we select the spectrum bin with the maximum SNR as the single snapshot of each of the receive antenna elements. And the antenna array vector is rewritten as . Thus, the covariance matrix of the receiving signals can be written as [27]where are the eigenvalues of , are their associated eigenvectors, and and . The columns of and are the eigenvectors associated with largest eigenvalues and smallest eigenvalues, respectively. For simplicity, we assume that the number of sources is known in this paper. With the known mutual coupling coefficients and the uncorrelated signals which have the high SNR, it is shown that the subspace spanned by , is approximately identical with the signal subspace spanned by and is approximately orthogonal to the noise subspace spanned by . According to MUSIC algorithm, we havewhere and are the searching angles and is zero whenever and the peaks of will correspond to the true DOAs.

However, if the mutual coupling coefficients are unknown, the performance of DOA estimation would be degraded and the peaks of would point to the error DOAs. Moreover, the different signals can be represented as one of them multiplied by a complex factor in the case of single snapshot, which can be taken as coherent sources. Therefore, the mutual coupling correction technique and the spatial smoothing technique [28] are used to improve the performance of DOA estimation in the following part.

The -axis and -axis antenna elements of the receive antenna array are divided as the subarrays separately. Denote the data vector of the th () subarray of the th row along -axis as , where is the number of the subarrays, and the data vector of the th () subarray of the th column along -axis as , where is the number of the subarrays. We haveAnd the covariance matrices are written aswhere , , and are the forward spatial smoothing covariance matrix, the backward spatial smoothing covariance matrix, and the forward-backward spatial smoothing covariance matrix of -axis, respectively. Similarly, , , and are the forward spatial smoothing covariance matrix, the backward spatial smoothing covariance matrix, and the forward-backward spatial smoothing covariance matrix of -axis. And is the “exchange” matrix, which is denoted asWe denote the equivalent equations of the data vector after applying the spatial smoothing technique aswhere , , , , and are the equivalent data vector, the equivalent mutual coupling coefficients matrix, the equivalent steering matrix, the equivalent signal vector, and the equivalent noise vector of -axis, respectively. Similarly, , , , , and are the equivalent data vector, the equivalent mutual coupling coefficients matrix, the equivalent steering matrix, the equivalent signal vector, and the equivalent noise vector of -axis.

In HF radar, the mutual coupling coefficients are unknown and the SNR is often not high. Therefore, the subspace spanned by is not orthogonal to the noise subspace spanned by , and the subspace spanned by is not orthogonal to the noise subspace spanned by . Then, we havewhere and are the residual items. Let denote the pseudoinversion, and then we can get the least square solution asBy using the original DOA estimates and obtained through the MUSIC algorithm, we can estimate the mutual coupling coefficients matrix by (13).

#### 3. An Online Particle Mean-Shift Approach

We have assumed that the additive noise is stationary, zero-mean Gaussian random processes. From (13), we know that the mutual coupling coefficients matrix can be obtained from the DOAs we estimated. Thus, each of the elements of the estimated mutual coupling coefficients matrix is a random process. Therefore, we can now formulate a tracking problem in a Bayesian context for the online estimation for the mutual coupling coefficients matrix. In this section, we complete the model by introducing the idea of the particle filter and the mean-shift approach.

From [29, 30], we know the residual items and in (13) can be represented as the linear combination of the additive noise. Thus, they are also stationary, zero-mean Gaussian random processes. And the bias of the estimates and is Gaussian distribution [29, 30]. Denote and as the number of the elements of the subarrays on the -axis and -axis separately. We wish to calculate the posterior probability distributions and which need the and dimension particles according to the idea of the particle filter algorithm [31] separately. To reduce the computational cost, (11) is rewritten aswhere and are Toeplitz matrices and and are the residual items which satisfyFor any and , we havewhere Substitute (15) into (16), and we have For each of the two steering vectors, such as , is a matrix, which can be expressed as follows [22]:According to (19), is also easily calculated, due to the fact thatwhere , , , and are the residual items with . And and are the noise items, where and are the factors. Define the coefficient matrices of the equations asThen, we havewhere and are the residual vectors, the th element of which is denoted asAs the additive noise is a stationary, zero-mean Gaussian random process, and are also Gaussian distribution, where . DenoteNote that the first elements of the vectors and are equal to 1, and (22) can be rewritten aswhereThus, the least square solution is easily obtained:In (27), we have reduced the computational cost from and dimensions to and dimensions. According to (27) and (20), we can use the idea of the particle filter to obtain the estimates of the mutual coupling coefficients matrix.

To reduce the computational cost further, (23) is approximately equal towhere and are the residual items. And substituting (28) into (20) and (22), we haveMoreover, (29) can also be rewritten aswhere and are the noise space of the covariance matrices and separately, which are the Toeplitz matrix form of and by the Toeplitz approximation transformation [32]. According to (24)–(27), we can also obtain the estimate of the mutual coupling coefficients matrix by (29) or (30).

For convenience, we denote the probability density functions of the th estimation by the idea of the particle filter as and . That is to say, we defineAnd and are the a posteriori probability density functions in and dimensionality space. To find a robust result of and , a mean-shift algorithm based on a kernel function is introduced in the following, which is a nonparametric density gradient estimation in nature. Conditions on the kernel function are derived to guarantee asymptotic unbiasedness, consistency, and uniform consistency of the estimates. This approach is a simple iterative procedure that shifts each data point to the average of data points around its neighborhood. And the mean-shift approach is applied to search the most rapid increase of the probability density function for the matrix tracking in the case of the estimated mutual coupling coefficients matrices generated by the particles.

Denote a new kernel function estimation of and [32, 33], and we havewhere is the number of the particles and and are the radius of the high-dimensional ball areas in the - and -dimensional Euclidean space separately. And the Epanechnikov kernel can be expressed as follows [33, 34]:Let The gradient estimation of and iswhere is the weight function. Equation (35) can be rewritten asDenoteThen, we use and to express the formula followed and , and we havewhere and are the estimations of kernel function G(**C**) for and and and are the mean-shift vectors. Then, (35) could be rewritten asWe can conclude from (40) that the vectors and are proportional to the gradient of and . Hence, and point in the direction of the most rapid increase of the probability density function.

The routine of the online particle mean-shift approach is shown in Algorithm 1. We refer to this algorithm as OPMA for convenience.

*Algorithm 1 (OPMA). *Initialize and as the identity matrices; this initial DOA’s estimates are calculated by the MUSIC algorithm with the covariance matrices obtained by (9). And the sums of the MUSIC spectrum amplitude of the peaks of the targets are denoted as and .

For , do (1)Let and in (29) and and in (30) be the random particles generated by (), where is a stochastic number with the uniform distribution within the range of to and is a quarter of the corresponding beamwidth.(2)Substitute the random particles into (29) and (30), and the coupling coefficient matrix estimates and are calculated by (27). Then, the MUSIC spectra are obtained by the noise spaces , , , and and the coupling coefficient matrix estimates. And the sums of the MUSIC spectrum amplitude of the peaks of the targets are denoted as , , , and .(3)If , let . Otherwise, let , where is a temporary variable. If , we reserve the results for the calculation of the mean-shift algorithm. Otherwise, the results are abandoned. Similarly, if , let . Otherwise, let , where is also a temporary variable. If , we reserve the results for the calculation of the mean-shift algorithm. Otherwise, the results are abandonedend for(4)The mean-shift results are calculated with the results selected in step (3) by (38). And we select the weights and in (38) as and .(5)Select the mean-shift results as the estimates of the equivalent mutual coupling coefficients matrix of -axis and -axis. Then, substitute into the MUSIC algorithm separately, and the estimates of the azimuth and the elevation of the targets can be obtained.

*Remark 2. * is an empirical value, which is the spatial spectrum amplitude of the MUSIC algorithm and can be about 10 dB. And the number of the particles can be about 20, which has the moderate computational complexity.

#### 4. Simulations

In this section, we evaluate the performance of the proposed approach mentioned above. Consider the receiver antenna array of a MIMO HF sky-wave radar system, which is a 2D uniform rectangle array with 50 identical and omnidirectional antennas in -axis and -axis separately, and the ratio of the distance between the neighboring elements and the wavelength is 0.5. Assume , and , , , , , , , , and . And the receiving targets have the same range which is 1000 km, and they have the same DOD and the same radial velocity. The snapshot is single one and the noise is the additive white Gaussian noise. Four approaches are considered in the following simulations, which are the spatial smoothing technique without the mutual coupling correction (which is referred to as SST), the spatial smoothing technique with the mutual coupling correction by (25) with and (which is the Conventional Mutual Coupling Coefficients Estimation Method and is referred to as CMCCE), the spatial smoothing technique with no mutual coupling, and the OPMA proposed in this paper. We consider the success probability, the estimation bias, and the estimation variance of the four approaches in the case of SNR = −10 dB~30 dB. 100 simulations are carried out for each different SNR. The estimation bias in each Monte Carlo simulation is measured by the root mean squared error (RMSE).

In the first simulation, we consider three targets with the different azimuths which are −4°, −1°, and 15° and the same elevation which is 90°. The success probability of the first simulation is shown in Figure 2. It is shown that OPMA has higher success probability than SST and CMCCE and lower success probability than the no coupling result in the low SNR region. The RMSE of the first simulation is shown in Figure 3. It is shown that OPMA has lower RMSE than SST and CMCCE and higher RMSE than the no coupling result. The variance of the first simulation is also calculated. The result is shown in Figure 4. It is shown that the proposed approach has smaller variance than SST and CMCCE.

In the second simulation, we consider three targets with the different elevations which are 0°, 3°, and 18° and the same azimuth which is 0°. The success probability of the second simulation is shown in Figure 5. It is shown that OPMA has higher success probability than SST and CMCCE and lower success probability than the no coupling result in the low SNR region. The RMSE of the second simulation is shown in Figure 6. It is shown that OPMA has lower RMSE than SST and CMCCE and higher RMSE than the no coupling result. The variance of the second simulation is also calculated. The result is shown in Figure 7. It is shown that the proposed approach has smaller variance than SST and CMCCE.

#### 5. Conclusion

We introduce the OPMA for the mutual coupling coefficients estimation of the 2D uniform rectangle array with the single snapshot in MIMO HF sky-wave radar. The equation of the mutual coupling coefficients matrix and the spatial smoothing technique of the uniform rectangle array are introduced. Besides, the equivalent equations of the mutual coupling coefficients matrix after the preprocessing of the spatial smoothing technique are derived. Based on the idea of the particle filter and the mean-shift algorithm, the OPMA is proposed, which has a moderate computational complexity. We compare the performance of the proposed approach with the no mutual coupling case and the approaches of SST and SMCCE. The proposed approach is more robust and more accurate than SST and SMCCE. The simulation results demonstrate the validity of the proposed method in terms of the success probability, the statistics of bias, and the variance.

#### Competing Interests

The authors declare that they have no competing interests.

#### Acknowledgments

This paper is supported by the National Natural Science Foundation of China (Grant no. 61301209).