#### Abstract

This paper addresses the issue of direction finding of a cyclostationary signal under impulsive noise environments modeled by *α*-stable distribution. Since *α*-stable distribution does not have finite second-order statistics, the conventional cyclic correlation-based signal-selective direction finding algorithms do not work effectively. To resolve this problem, we define two robust cyclic correlation functions which are derived from robust statistics property of the correntropy and the nonlinear transformation, respectively. The MUSIC algorithm with the robust cyclic correlation matrix of the received signals of arrays is then used to estimate the direction of cyclostationary signal in the presence of impulsive noise. The computer simulation results demonstrate that the two proposed robust cyclic correlation-based algorithms outperform the conventional cyclic correlation and the fractional lower order cyclic correlation based methods.

#### 1. Introduction

Arrays of sensors such as radio antennas can be used to detect the presence of propagating signals and estimate their directions of arrival (DOA) and other parameters. Their applications have been found in many areas, for example, radar, sonar, biomedical signal processing, and communication systems [1, 2]. Conventional array processing methods generally exploit spatial properties of the signals impinging on an array of sensors. In applications for radar, sonar, or telecommunications there are many man-made signals whose cyclostationary properties can be used to cancel interference and background noise [3]. The earliest approach was proposed by Gardner [4] who introduced the concept of cyclostationarity into array signal processing to suppress interference and noise. Including this property in signal processing algorithm design can improve the performance of existing algorithms, especially the DOA estimation algorithms. Several algorithms have been proposed in the literature along this line [5–7]. Instead of using the correlation matrix as being done in conventional methods, these cyclostationarity–based algorithms require estimating the cyclic correlation (CCO) matrix to reflect the cyclostationarity of incoming signals which can be one of the following three cases: (1) having baud rates or (2) being modulated by a carrier signal in the way that they are used in radar and radio communication applications or (3) both.

One common assumption made by conventional methods and cyclostationarity methods is that the ambient noise is assumed to be Gaussian distributed and can be characterized by only second-order statistics (SOS). However, in many real world applications the noise often exhibits non-Gaussian properties and sometimes is accompanied by strong impulsiveness [8]. For example, natural sources such as atmospheric noise resulting from thunder storms, car ignitions, microwave ovens, and other types of man-made signal sources generally result in aggregating noises that may produce high amplitudes during small time intervals. To address this type of noise the *α*-stable distribution was proposed as a better and suitable noise model [9]. It has been also shown to have potential in characterizing various impulsive noises via selecting different values of the parameter *α*.

Since *α*-stable distribution has no finite SOS, the SOS-based estimation of signal parameters, such as DOA, is generally not applicable. Therefore, to address this issue the fractional lower order statistics (FLOS) was recently proposed [10, 11] such as the fractional lower order moments (FLOM) [11] and the phased fractional lower order moments (PFLOM) [10, 11]. However, FLOS requires a priori knowledge of *α*-stable distribution, which is difficult to estimate in some practical applications. In order to measure similarity between two random variables, correntropy has been proposed and successfully applied in *α*-stable signal processing [12]. By virtue of correntropy, [13, 14] defined a robust correlation criterion, the correntropy-based correlation (CECO) to estimate DOA by MUSIC, which can be called CECO-MUSIC algorithm. Recently, [15] also defined a robust correlation, called nonlinear transform correlation (NTCO), by introducing a nonlinear compression function into correlation. This paper developed a novel algorithm for DOA estimation in impulsive noise, to be called NTCO-MUSIC which uses the NTCO matrix to replace the correlation matrix of the common used by MUSIC algorithm.

To further handle the cyclostationary signals under *α*-stable impulsive noise environment, You et al. defined the fractional lower order cyclic correlation (FLOCC) [16, 17] and the phased fractional lower order cyclic correlation (PFLOCC) [18] for the DOA estimation. Liu et al. also defined the fractional lower order cyclic cross-ambiguity function [19] for joint estimation of time difference of arrival and frequency difference of arrival for cyclostationary signals under *α*-stable impulsive noise.

In this paper, we introduce two robust cyclic statistics based on CECO and NTCO for cyclostationary signal, called the correntropy-based cyclic correlation (CECCO) and nonlinear transform based cyclic correlation (NTCCO), both of which can be implemented in conjunction with the MUSIC algorithm for DOA estimation of cyclostationary signals under *α*-stable impulsive noise environment. Specifically, the MUSIC algorithm uses CECCO or NTCCO matrix of signals received from the array which are called CECCO-MUSIC and NTCCO-MUSIC algorithms. To demonstrate the performance of the proposed two methods to the FLOCC-based MUSIC algorithm (FLOCC-MUSIC) and cyclic correlation-based MUSIC algorithm (CCO-MUSIC), computer simulation experiments are conducted for comparative study and analysis.

#### 2. *α*-Stable Distribution Specified Noise Model

This section describes a noise model specified by *α*-stable distribution with its characteristic function specified bywhere *γ* and are the dispersion and location parameters, respectively, and is defined byand the sign function, , is given asIn particular, *α* () is the characteristic exponent that measures the thickness of the tails of the distribution where the smaller *α* is, the thicker its tails are. Also, *β* is the symmetry parameter, if *β* = 0*,* the distribution in which case the observation is referred to as the symmetry *α*-stable (S*α*S) distribution. When *α* = 2 and *β* = 0, the *α*-stable distribution becomes a Gaussian distribution. An important difference between the Gaussian and the *α*-stable distribution is that the former has only first two moments while the latter does not have any statistics when the moments of order are greater than or equal to *α*.

#### 3. Problem Formulation and CCO-Based Method

##### 3.1. Problem Definition

Suppose that there is a uniform linear array (ULA) of antennas and electromagnetic waves impinging on the array from angular directions where the incident waves are also assumed to be far-field narrowband point sources. In this paper, signals of interest (SOIs) are further assumed to be cyclostationary signals with cycle frequency *ε* (with ), and all of the remaining signals are referred to as signals of no interest (SONIs), which either have different cycle frequencies or are not cyclically correlated with SOIs. Furthermore, the noise is also assumed to be i.i.d and is not correlated with signals. Based on the above assumptions, the signal received from the sensor in the array with the complex envelope representation is given bywhere is the response of the sensor with respect to which is the signal emitted by the th source with the cycle frequency and represents all SONIs plus noise received by the sensor.

Now we assume that an observation vector received from the array is denoted by and expressed aswhere contains the SOIs and represents SONIs plus noise. Let be the matrix made up of steering vectors of the arriving SOIs with their steering vector given bywhere is the carrier wavelength of all SOIs and is the interspacing.

##### 3.2. CCO-Based Method

Under the Gaussian noise assumption, for the cyclic frequency and some lag parameter *τ*, the CCO matrix of the received data vector is defined byInstead of using the eigenvalue decomposition (EVD) to decompose (7), the DOA estimation algorithm uses the singular value decomposition (SVD) to decompose (7) as intowhere the subscripts and stand for signal and noise subspaces, respectively, and are unitary matrices, and the diagonal elements of the diagonal matrices and are arranged in the decreasing order. Specifically, the diagonal elements of tend to approach to zero as the number of samples increases to infinity. Thus, the CCO-based MUSIC algorithm, defined as CCO-MUSIC, can estimate DOA by searching for the peaks of the following spatial spectrum:

#### 4. FLOCC-Based Method

Despite the fact that the CCO_MUSIC algorithm has been shown to be effective in high-resolution direction finding under the Gaussian noise assumption, it cannot be applied to *α*-stable distributed random processes because CCO does not have finite variance in these processes. To deal with this issue, the FLOCC statistics was proposed to be implemented with MUSIC algorithm to obtain the DOA estimations of SOIs [16, 17]. The resulting algorithm is called the FLOCC-MUSIC algorithm.

In the FLOCC-MUSIC algorithm, the CCO matrix defined in (7) was replaced by the FLOCC matrix , defined bywhere is the order of the fractional lower order moment and . For a complex process , . If is expressed in the form of polar coordinates as , it is easy to derive which can be used to suppress the amplitude of the impulsive noise rather than the phase *θ*. So, the cyclic frequency defined by the second-order cyclic statistics is also suitable for the FLOCC.

#### 5. Robust Cyclic Correlation

Although FLOCC can effectively suppress the *α*-stable impulsive noise contained in the cyclostationary signal, it needs to know the characteristic exponent of the *α*-stable distribution in advance which is difficult to estimate in practical applications. So, in this paper we introduce two new robust cyclic correlation functions which can be used under *α*-stable impulsive noise environment without knowing the characteristic exponent of *α*-stable distribution.

##### 5.1. Correntropy-Based Cyclic Correlation

By taking advantage of CECO and Gaussian kernel, a new cyclic statistics for two random variables and can be defined as an effective alternative to the conventional cyclic correlation which can be used in *α*-stable impulsive noise environment, to be called correntropy-based cyclic correlation (CECCO) as follows: where is a given positive constant and is the scale parameter. It was shown in [13] that the CECO behaved like a correlation in Gaussian noise as well as a robust -estimation correlation in impulsive noise environment. Similarly, the CECCO also behaves like a cyclic correlation in Gaussian noise and a robust -estimation cyclic correlation in impulsive noise environment.

##### 5.2. Nonlinear Transform Based Cyclic Correlation

By applying a nonlinear transform to cyclic correlation, we can define a nonlinear transform based cyclic correlation (NTCCO) as follows:where is called scale factor.

Equations (11) and (12) can be further combined as where and is a scalar quantity where is used to suppress the amplitude of the correlation between random variables and and does not affect their periodicity. So, the robust cyclic correlation function defined by (13) has the same cyclic frequency as CCO. In the meantime it also has a good inhibition effect on the amplitude of the impulsive noise.

##### 5.3. Simulation of the Robust Cyclic Correlation

For *α*-stable impulsive noise the commonly used the signal-to-noise ratio (SNR) is not applicable because the noise variance is not finite. Since the dispersion of *α*-stable distribution is characterized by the parameter , we use the generalized signal-to-noise ratio (GSNR) [11] here, which is expressed aswhere is the variance of the signal.

There are two time varying AM signals of interest with the *α*-stable impulsive noise for defined bywhere is the same amplitude for both carriers, is the amplitude modulation factor, is the initial phase, is the modulating frequency, and is the carrier frequency. Set , , and sample frequency . Figure 1 shows the CCO, FLOCC, CECCO, and NTCCO of the two time varying AM signals defined by (15) with the characteristic exponent of the impulsive noise and the GSNR = 2 dB.

**(a)**

**(b)**

**(c)**

**(d)**

It can be seen from Figure 1 that CCO cannot represent the cyclic statistics characteristics of the two signals accurately at the cyclic frequency and , and in the meantime it has nonzero values at noncyclic frequency because of the infinite second-order moments of the *α*-stable noise. Although FLOCC can demonstrate the cyclic statistics characteristics at the cyclic frequency, it also has some nonzero values at noncyclic frequency. Unlike CCO and FLOCC, CECCO and NTCCO not only can demonstrate the sharply cyclic statistics characteristics at the cyclic frequency, but also have very close to zero values at noncyclic frequency. Accordingly, it is expected that a DOA estimation algorithm based on CECCO and NTCCO will be superior to algorithms designed based on CCO and FLOCC, a fact that will be verified by the simulations in Section 8.

#### 6. DOA Estimation Based on Robust Cyclic Correlation

Using CECCO, we can define a CECCO matrix of a signal received by an array described by (4) where its element is defined asApplying SVD to the matrix and formulating the corresponding spatial spectrum to obtain the DOA estimates of SOIs yields the CECCO-MUSIC algorithm.

In analogy with the CECCO-MUSIC algorithm we can also derive the NTCCO-MUSIC algorithm using the NTCCO matrix defined as

#### 7. Discussions

The correntropy [12] is a local similarity measure between two arbitrary random variables based on the Gaussian kernel function. The cyclic correntropy (CCE) which was proposed in [20] was a generalized correntropy for cyclostationary signals. The correntropy-based correlation (CECO) defined in [13] is an effective substitute for conventional correlation functions that were used for DOA estimation in an *α*-stable impulsive noise. This paper generalizes the CECO concept for cyclostationary signals, to be called the correntropy-based cyclic correlation (CECO) which can be used for DOA estimation of the cyclostationary signals in an *α*-stable impulsive noise. In addition to CECCO another new concept of a robust cyclic correlation called nonlinear transform cyclic correlation (NTCCO) is also introduced in this paper for the DOA estimation of the cyclostationary signals by incorporating a nonlinear compression function into cyclic correlation. According to our experiments it turns out that the NTCCO-based method performs better than the CECCO-based method.

There are also other methods to estimate the DOA in the impulsive noise environment, such as the data-adaptive zero-memory (DA-ZM) algorithm in [22], the expectation-maximization (EM) algorithm in [21], and the FLOS based methods in [16–18]. These three methods along with our proposed CECCO and NTCCO-based DOA estimation method are all subspace-based algorithms in the sense that the subspace can be obtained by the SVD of the robust correlation matrix calculated by signals received by the array. However, there is also a significant difference between the three DOA estimation methods, DA-ZM, EM, and FLOS, and our proposed methods, which is how the robust correlation matrix of the array received signals is calculated. In the three previously mentioned DOA estimation methods, the data vectors received by the array are first preprocessed by a zero-memory nonlinearity process to suppress the impulsive noise. It then uses the correlation matrix of the preprocessed received data vectors as the robust correlation matrix estimation of the array received signals. It is a two-stage sequence process with the first stage process of impulsive noise suppression followed by the second stage process of correlation matrix estimation. Such resulting process is called sequential suppression and correlation (SQSC) process. In our two proposed CECCO- and NTCCO-based methods both the suppression of the impulsive noise and the correlation matrix estimation are carried out simultaneously. The resulting process is referred to as simultaneous suppression and correlation (SMSC) process. The simulation results demonstrate that the performance of SMSC is better than SQSC.

When a source signal is cyclostationary, the methods in [21, 22] do not utilize the cyclostationarity of the source signal. In other words, the methods in [21, 22] do not have signal selectivity. By contrast, our proposed two methods utilize the cyclostationarity of source signals. Accordingly, our proposed methods not only can suppress the impulsive noise in the array received signal but also have signal selectivity.

Finally, Table 1 summarizes the above discussions by comparing various signals and different functions used to process the array received signals vector and the correlation matrix of the array received signals.

#### 8. Simulation Results

This section conducts computer simulation experiments to compare the relative performance of our proposed CECCO-MUSIC and NTCCO-MUSIC algorithms to CCO-MUSIC and FLOCC-MUSIC algorithms under the S*α*S impulsive noise environment. Two criteria are used to evaluate their performances. One is the probability of resolution. In doing so, a popular resolution criterion is used and defined by the following threshold equation [11]:where and are the angles of arrival of the two SOIs and is the mid-point between and . The two SOIs are said to be resolvable if (18) holds. Two hundred independent Monte Carlo experiments were simulated. Let be the number of pairs of two incident angles that can be resolved. The probability of resolution is then defined as . In case two SOIs can be resolved by the th Monte Carlo experiment, is then set to the estimation of . With this definition, the average mean square error (MSE) of the DOA estimation is then defined as

Suppose that a ULA consists of ten sensors with an interspacing of half a wavelength and the incoming signals are uncorrelated binary phase-shift keying- (BPSK-) modulated sources. We further assume that sample frequency is , and the carrier frequency of the BPSK SOIs is . Other signals are considered as interference with a carrier frequency of . In what follows, the cycle frequency was simulated by which was usually twice the carrier frequency . The four algorithms, CCO-MUSIC, FLOCC-MUSIC, CECCO-MUSIC, and NTCCO-MUSIC algorithms, were evaluated for performance comparison.

*Simulation 1*. Suppose that there are one SONI arriving from and two BPSK SOIs coming from and , respectively. The noise was S*α*S distributed with . The GSNR is set to GSNR = 20 dB, and the number of snapshots is 600. Figure 2 plots the spatial spectrum of the four versions of the MUSIC algorithm, CCO-MUSIC, FLOCC-MUSIC, CECCO-MUSIC, and NTCCO-MUSIC algorithms, where the CCO-MUSIC algorithm failed to separate the two DOAs of the SOIs, while the FLOCC-MUSIC algorithm could separate the two DOAs of the SOIs, but its estimate was not very accurate. By contrast, our proposed algorithms, the CECCO-MUSIC and NTCCO-MUSIC algorithms, not only successfully separated the two DOAs of the SOIs and but also correctly estimated DOA with very high accuracy. In particular, the NTCCO-MUSIC algorithm has a much sharper spatial spectrum than that generated by the CECCO-MUSIC algorithm.

*Simulation 2*. Figure 3 plots probabilities of resolution and MSEs produced by the four test MUSIC algorithms with various GSNRs. The SOIs’ angles of arrival were and , and the SONI’s angle of arrival was set to . The characteristic exponent of the *α*-stable impulsive noise was . The snapshots number used by the simulation was 600. As we can see from Figure 3 the performance of NTCCO-MUSIC and CECCO-MUSIC algorithms was much better than that produced by FLOCC-MUSIC and CCO-MUSIC algorithms. Especially, the best one was NTCCO-MUSIC algorithm which produced the highest probabilities of resolution with more than 90% of success and the least MSE in all GSNRs. The worst one was the CCO-MUSIC algorithm which nearly failed and produced largest MSE when GSNR < 14 dB.

**(a)**

**(b)**

*Simulation 3*. Figure 4 plots probabilities of resolution and MSEs produced by the four algorithms varying with different values of the characteristic exponent of the *α*-stable impulsive noise. The SOIs’ angles of arrival were and , and the SONI’s angle of arrival was set to , the number of snapshots used by the simulation was 600, and GSNR = 14 dB. As shown in Figure 4 NTCCO-MUSIC and CECCO-MUSIC algorithms demonstrated their performance enhancement over FLOCC-MUSIC and CCO-MUSIC algorithms in the sense of both probability of resolution and MSE. Moreover, the performance of NTCCO-MUSIC algorithm is slightly superior to CECCO-MUSIC algorithm.

**(a)**

**(b)**

*Simulation 4*. Figure 5 plots probabilities of resolution and MSEs produced by the four algorithms by changing the number of snapshots. The SOIs’ angles of arrival were and , and the SONI’s angle of arrival was . The characteristic exponent of impulsive noise was set to and GSNR = 14 dB. As demonstrated in Figure 5 the performance of all four methods was improved as the number of snapshots increased. Nevertheless, our proposed NTCCO-MUSIC algorithm and CECCO-MUSIC algorithm produced lower MSEs and higher probabilities of resolution compared to the other two MUSIC algorithms, CCO-MUSIC algorithm and FLOCC-MUSIC algorithm, when the same number of snapshots was used.

**(a)**

**(b)**

*Simulation 5*. Figure 6 plots probabilities of resolution and MSEs produced by the four algorithms changing with various degrees of the angular separation. The angles of arrival of the SOIs were and , where is the angular separation of the two SOIs and varied from to in a step size of . The DOA of SONI was . The characteristic exponent of impulsive noise was set to and GSNR = 14 dB. From Figure 6 the proposed NTCCO-MUSIC algorithm and CECCO-MUSIC algorithm were superior to CCO-MUSIC and FLOCC-MUSIC algorithm in the sense that the probability of resolution of the NTCCO-MUSIC algorithm almost reached 100%, while the probabilities of resolution of CCO-MUSIC and FLOCC-MUSIC algorithms were lower than 20% when the angular separation was .

**(a)**

**(b)**

#### 9. Conclusion

This paper proposes two robust cyclic correlation functions, correntropy-based cyclic correlation (CECCO) and the nonlinear transform based cyclic correlation (NTCCO). By virtue of CECCO and NTCCO we can implement the CECCO-based and NTCCO-based array received signals matrix in conjunction with the MUSIC algorithm to obtain DOA estimates of SOIs. Computer simulation results demonstrate that CECCO-MUSIC and NTCCO-MUSIC algorithms indeed outperform FLOCC-MUSIC and CCO-MUSIC algorithms in *α*-stable impulsive noise environments. In addition, experiments also show that NTCCO-MUSIC algorithm performs slightly better than CECCO-MUSIC algorithm.

#### Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

#### Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61301228 and 61371091 and the Fundamental Research Funds for the Central Universities under Grant 3132016331.