Abstract

Weak signal detection is a significant problem in modern detection such as mechanical fault diagnosis. The uniqueness of chaos and good learning ability of neural networks provide new ideas and framework for weak signal detection field. In this paper, Elman neural network is applied to detect and recover weak pulse signal in chaotic noise. For detection problem of weak pulse signal under chaotic noise, based on short-term predictability of chaotic observations, phase space reconstruction for observed signals is carried out. And Elman deep learning adaptive detection model (EDAD model) is established for weak pulse signal detection, and a hypothesis test is used to detect weak pulse signal from the prediction error. For the recovery of weak pulse signal under chaotic noise, a double-layer Elman deep neural network recovery model (DEDR model) is proposed, which is based on the Elman deep learning network model and single-point jump model for weak pulse signal, and it is optimized with goal of minimizing mean square prediction error of the Elman model. The profile least squares method is applied to estimate parameters of the DEDR model for difficult recovery of weak pulse signal because the DEDR model is essentially a semiparametric model with parametric and nonparametric parts. In the end, simulation experiments show that the model built in this paper can effectively detect and recover weak pulse signal in the background of chaotic noise.

1. Introduction

Weak signal is a weak amount that is difficult to detect. It is small amplitude compared to background noise and is a signal that is often annihilated by noise and has a low signal-to-interference ratio [1, 2]. Weak signal detection has been used for various methods to detect useful signals from obseration signals, and it has broad application prospects in the fields of radar, communication, automation, fault diagnosis, and seismic monitoring [37].

With the rapid development of science and technology, it is urgent to detect and recover weak signals in engineering applications. At the same time, with maturity and wide application of chaos theory, the combination of chaos theory and detection and recovery of weak signals has become a research trend.

In 1990, Leung and Haykin first introduced chaos theory into the field of sea clutter and then succeeded in extracting weak target signals from chaotic noise background based on chaotic characteristics using neural networks and MPSV methods [812]. In recent years, many scholars have done a lot of research on the weak signal under chaotic noise and put forward many effective methods. For example, Wang et al. studied a local tangent space projection method that extracts weak harmonic signals from strong chaotic interference [13]. Liu et al. proposed a method to detect weak harmonic signals from fractal noise by combining multiscale fuzzy adaptive Kalman filtering with duffing chaotic oscillators [14, 15]. Kurian and Leung proposed a method combining dynamic reconstruction and the chaotic synchronization enhancement model to detect weak signals in chaotic clutter [16]. Du and Hou proposed the SVM method for detecting weak harmonic signals in chaotic background [17]. Xing et al. proposed a hybrid algorithm for weak signal detection in the context of chaotic sea clutter [18]. Li et al. proposed a local linear-periodic detection-Kalman filtering hybrid algorithm to detect weak signal in strong chaotic backgrounds [19]. Su et al. proposed a local linear-periodic graph detection-Kalman filtering hybrid algorithm for the sinusoidal signal problem under strong chaotic background [20]. Su et al. proposed a local linear autoregressive model and a double local linear model to detect and recover weak pulse signals in the background of chaotic noise [21]. These methods generally can effectively detect target signals, but often have problems such as low sensitivity, poor adaptability, or high threshold value for SNR of detected weak signals.

With the continuous research on neural networks and deep learning, scholars combined neural network methods based on chaos theory and phase space reconstruction, which provides a new direction for improving the accuracy of signal detection. Leung and Lo used an RBF network predictor to introduce a detection technique for small sea targets based on this dynamic model [9]. Chinese scholars He et al. proposed a method for detecting signals submerged under chaotic background using neural networks and studied the antinoise interference of this method [22]. Zheng et al. proposed an echo state network method for weak signal detection in chaotic background and used genetic algorithm to optimize its model parameters [23].

The basic idea of the artificial neural network method is to use the outstanding learning ability and good nonlinear approximation ability of feed forward network and use chaotic background signal generation mode to train feed forward network, which makes it better than the traditional linear prediction method in predicting highly complex nonlinear time series. After training, the received signal is subjected to single-step prediction, and the predicted value is compared with actual received value to obtain a prediction error, the prediction error is processed, and finally the presence or absence of signal is obtained.

As a typical local regression network, Elman neural network belongs to feedback neural network. It is very similar to the forward neural network, but it has stronger computing power. Its outstanding advantage is that it has strong optimization calculation and associative memory function. Cao and Wang [24] first proposed the use of Elman network to detect transient signals and periodic signals under chaotic background, but only by visual inspection of the detection performance and no further recovery of the detection signal. Based on this, our paper considers an adaptive detection and recovery of Elman deep learning network based on weak pulse signal in the background of chaotic noise. In the first part, we reconstruct phase space of observed signal and construct the Elman deep learning network to obtain one-step prediction error. The signal detection problem is converted into a hypothesis test problem for prediction error and then is combined with ACC value, ROC curve, and AUC value, and dynamic threshold adaptive detection detects the presence of weak pulse signal. In the second part, a double-layer Elman deep neural network recovery model is constructed based on pseudo-observation signal reconstruction, and parameters of the model are estimated by using the least squares method to recover the weak pulse signal. Specific idea is shown in Figure 1:

The structure of this paper is as follows: Section 2 describes weak signals for adaptive detection; Section 3 recovers weak pulse signals based on DEDR model; Section 4 presents simulation experiments and analysis; and Section 5 concludes the paper.

2. Detection of Weak Pulse Signals in the Background of Chaotic Noise

2.1. Detection of Weak Pulse Signals

The problem of detecting weak pulse signals from the background of chaotic noise can be abstracted into the following hypothesis test problems:where represents observed signal, represents chaotic noise background signal, represents a weak pulse signal and is independent of chaotic noise background signal , represents white noise with a mean of 0, and represents the sum of chaotic noise background signal and white noise .

Since weak pulse signal is submerged in chaotic noise background signal , if the hypothesis test is directly performed by equation (1), it is impossible to detect whether or not observation signal contains . Therefore, we must first remove the interference of chaotic noise background signal and convert equation (1) into the following hypothesis test problem:

That is, work to be done in this section is as follows: (1) using the prior knowledge of chaotic background signal, a single-step prediction model of observed signal is established to obtain prediction error; (2) detecting whether there is a weak pulse signal from prediction error.

2.2. Elman Deep Learning Neural Network Model

Elman deep learning neural network is a typical dynamic recurrent neural network. It is based on basic structure of BP network, adding a receiving layer in hidden layer as a one-step delay operator to achieve the purpose of memory so that the system has the ability to adapt to time-varying characteristics and enhance the global stability of the network. It has more computational power than feed forward neural network and can also be used to solve the problem of fast optimization.

Elman neural network is generally divided into four layers: an input layer, a hidden layer, a receiving layer, and an output layer. The connection between its input layer, hidden layer, and output layer is similar to a feed forward network. The unit of input layer only serves as a signal transmission, and the output layer unit plays a weighting role. The hidden layer unit has two kinds of activation functions, linear and nonlinear. Usually, activation function takes sigmoid nonlinear function. The receiving layer is used to memorize the output value of hidden layer unit before it can be regarded as a delay operator with one-step delay. The output of hidden layer is automatically connected to the input of the hidden layer by the delay and storage of the receiving layer. This self-association method makes it sensitive to historical data. The addition of an internal feedback network increases the ability of network itself to process dynamic information to achieve dynamic modeling. Its structure is shown in Figure 2.

In Figure 2, a set of input attributes is known as , where , . Each connection connected to a neuron has a corresponding weight , where the connection weight is the connection weight matrix of the receiving layer unit and the hidden layer unit, is a connection weight matrix of the input layer unit and the hidden layer unit, and is a connection weight matrix of the hidden layer unit and the output layer unit. and represent the outputs of the receiving layer unit and the hidden layer unit, respectively, represents the output of the output unit, and is the self-joining feedback gain factor.

Let , , and .

As shown in the structure diagram, the mathematical model of Elman neural network is

The function is the activation function of output neuron; the function is called the activation function of hidden layer neurons. The activation function often selects sigmoid function [25], which is an “S” type differential activation function. Its output is 0∼1, as shown in the following formula:

The general expression obtained by (3)–(6) is

The Elman neural network learning process is shown in Figure 3.

Let the actual output of -th system be ; then, the objective function of Elman neural network, i.e., the error function, can be expressed as

According to the gradient descent method, the partial derivative of for the weight is calculated and made 0, and the learning algorithm of Elman neural network can be obtained:where , , and are the learning rates of , , and , respectively.

Therefore, the gradient descent method of the momentum term is obtained as follows:where and is the momentum term coefficient, which is generally 0.9.

2.3. Building a Single-Step Predictive Model

The steps of establishing a single-step prediction model for the observed signals are as follows: (1) phase space reconstruction of the observed signal; (2) establishing an Elman deep learning network for the observed signal and obtaining a single-step prediction error; and (3) dynamic threshold adaptive detection.

2.3.1. Phase Space Reconstruction

For the observed signal , a phase point in the reconstructed phase space can be expressed as , where , . The Takens theorem [26] states that for each point in the reconstructed phase space trajectory, there is a smooth map such that , . If you can find or find the approximate mapping of , you can predict the next data point . In this paper, the complex autocorrelation method [27] is used to solve the delay time , and the Cao method [28] is used to solve the embedding dimension .

2.3.2. Establishing an Elman Deep Learning Adaptive Detection Model (EDAD Model)

Establish an approximate mapping of the observed signal after reconstructing the phase space :

The single-step prediction value is obtained, and the prediction error is also obtained:

Substituting equation (9) into equations (3) and (5), the prediction residual can be obtained as

2.3.3. Dynamic Threshold Adaptive Detection (DTAD)

Let , be a sample with an overall capacity of , with a hypothesis test:

Construction statistic: . When is established, T obeys the t-distribution with degree of freedom . For a given , a value = , where . When , the original hypothesis is rejected. At this time, it can be considered that the weak pulse signal is included in the observed signal, and the threshold value of the test can be obtained from the t-distribution table, and the dynamic threshold adaptive detection can be made according to the threshold value, thereby making the signal detection result more objective.

2.4. Detector Performance Evaluation

A hypothesis test is performed according to Section 2.3 to determine whether there is a weak pulse signal in the observation signal . The flow of detecting weak pulse signals using the Elman deep learning network model is shown in Figure 4.

The weak pulse signal has been detected using the Elman deep learning network model in Section 2.3, but the detected signal is only judged from the error graph by the naked eye, which makes our judgment highly likely to cause misjudgment, so we envision finding ways to make an objective diagnosis.

2.4.1. Accuracy

The scientific definition of accuracy (Acc) refers to the degree to which the average value of multiple measurements under certain experimental conditions is consistent with the true value, expressed as error. It is used to indicate the magnitude of the systematic error. The absolute value of the measurement error is large, and its accuracy is low. But the accuracy is not equal to the error.

Assuming that a signal is detected as 1 and no signal is detected as 0, the four cases that occur are as shown in Table 1:

It can be seen from Table 1 that TP and TN are the probabilities of correct prediction and can be obtained:

2.4.2. ROC Curve and AUC

Receiver operating characteristic curve (ROC curve) is also called the sensitivity curve. ROC curve is based on a series of different two-category methods (demarcation value or decision threshold), with the true-positive rate (TPR) as the ordinate and the false-positive rate (FPR) as the abscissa. The true-positive rate (TPR) is calculated as , and the false-positive rate (FPR) is calculated as .

The ROC curve evaluation method has a wide range of application. In practical applications, the closer the ROC curve is to the upper left corner, the higher the accuracy of the test is. The point closest to the ROC curve in the upper left corner is the best threshold with the least error, with the fewest false positives and false negatives.

Since the ROC curve does not clearly indicate which classifier is better, the AUC is used here for further measurement. AUC (area under the curve) is defined as the area under the ROC curve. Obviously, the value of this area will not be greater than 1. Since ROC curve is generally above the line y = x, the range of AUC is usually between 0.5 and 1. AUC can be used as a numerical value to directly evaluate the quality of the classifier. The value is larger, and the performance is better. Let M and N be the number of positive samples and the number of negative samples, respectively. indicates the sequence number of the i-th sample (the probability score is sorted from small to large, ranked at the i-th position), and indicates that only the sequence numbers of the positive samples are added together, that is, the formula is obtained as

3. Recovery of Weak Pulse Signals in the Background of Chaotic Noise

3.1. Double-Layer Elman Deep Neural Network Recovery Model (DEDR Model)

A single-point jump model for weak pulse signals is established based on the characteristics of weak pulse signals: , where , , .

So to restore , we only need to estimate .

According to Section 2, if a weak pulse signal is detected, the following model is established in combination with the weak pulse signal:

The above model is called double-layer Elman deep neural network recovery model (DEDR), where , represents white noise with zero mean. It can be seen from equation (17) that the model contains two parameters and . To recover , only needs to be estimated. Therefore, the problem of recovering the weak pulse signal in the background of chaotic noise is transformed into the problem of estimating the parameters in the model.

3.2. Parameter Estimation of DEDR Model

When the squared sum of the errors of the model is the smallest, the value of is estimated to be the optimal value, namely,

As can be seen from equation (18), the value of is estimated to require a value of . However, from equation (17), it is known that the value of is estimated to require a value of . Here, the profile least squares (PLS) method [29] is selected to estimate the values of and simultaneously.

The procedure for estimating the optimal value of the parameter using PLS algorithm is as follows: (1) estimating the value of given the value of ; (2) updating the value of with estimated in the first step; (3) repeating steps (1) and (2) until the best estimate is obtained. The specific calculation process is as follows:(1)Given the value of , estimate the value of (set initial value to 0).As can be seen from equation (3), . The analogy (18) formula establishes the estimation equation for :When is given, the Elman deep neural network model can be established by the momentum gradient descent method and can be calculated according to equation (19).(2)Update the value of Suppose that there isAssuming that is the smallest solution of the above equation, , where represents the derivative of .For any given near , there is an approximation: is a Hessian matrix of . The above observation leads to a one-step iterative estimate:(3)Cycle through steps 1 and 2 until the two consecutive values obtained by equation (23) are equal or the difference is very small and the value at this time is the optimal solution.

3.3. Recovery of Weak Pulse Signals

The process of estimating the weak pulse signal in the background of chaotic noise is shown in Figure 5. The pseudocode is given in Algorithm 1.

Inputs: observation signal , parameters , etc.
Outputs: optimal value of parameters and , weak pulse signal .
Begin
Phase space reconstruction after normalization of observation data
 Set initial parameter values:
 While ()
  Checked by Section 2 for the presence of ;
  Construct a DEDR model according to (18);
  Estimating the value of from (19);
 Update the value of using the profile least squares method;
 Substituting the estimated value of into the model and calculating ;
End while
Print optimal parameter and , calculation
End
Note: has a weak effect on the phase space reconstruction. When iterating , its delay time and embedding dimension are unchanged.

4. Simulation Experiment Results and Analysis

In order to verify the feasibility and effectiveness of the Elman deep learning network proposed in this paper, simulation experiments were carried out. In this paper, the Lorenz system is used to generate the chaotic noise background signal, the detection threshold is measured by the signal-to-interference ratio (SIR) [30, 31], and the mean square error (MSE) and ROC curve are used to measure the accuracy of the recovery result.where , , , and are the mean values of and , and are the energy of signal and , and is the variance of white noise .

The Lorenz system equation is as follows:where are time functions and the parameter . Assuming that the initial condition and the sampling time seconds, 10,000 data points are generated by the fourth-order Runge–Kutta method, and the first component is taken as the chaotic noise background as . Round off the first 3,000 points (ensure that the system fully enters the chaotic state) and select 4,000 consecutive sequences as the chaotic noise background, denoted as . The delay time and the embedding dimension of are determined by the complex autocorrelation method and Cao’s method.

4.1. Experiment 1: Detecting the Presence of Weak Pulse Signals

Consider a weak pulse target signal composed of eight pulse signals, i.e., , where , , . Generate a time series of length 4000, recorded as . The results of signal detection using the Elman deep learning network are shown in Figure 6.

Figures 6(a) and 6(b) show the time diagram of chaotic noise background signal and observed signal . It can be seen from Figure 6 that the weak pulse signal has a weak influence on the chaotic background signal, which basically cannot be observed from the graph, so the embedding dimension and delay time of and should be the same using the same method. Figure 6(c) gives the prediction error graph for . It can be seen intuitively that there is a significantly larger prediction error value, which means that there may be weak signals. Judging by the method of Section 2.3, there is indeed a weak pulse signal in the observed signal , which is the same as the illustrated result

4.2. Experiment 2: Model Performance Testing

From the results of Experiment 1, it can be known that the Elman model detects weak signals other than in , so it is necessary to detect whether the detected weak pulse signal detection is true and effective. Figure 7(a) gives the error function graph of the Elman deep learning model. The error of the graphic display is very rapid; after 500 iterations, the error curve becomes very gentle, indicating that the neural network at this time has achieved a good learning effect. Figure 7(b) gives ROC curve of the model. The ROC curve converges quickly and approaches the upper left corner. It can be seen that Elman deep learning network model works very well. At the same time, SIR reaches −85.21699, and MSE is 0.018904, which is smaller than 0.1. It also shows that Elman deep learning network has a good detection effect on weak pulse signal. However, this is only a test result obtained by subjective judgment of the graph, and the probability of misjudgment is large.

Here, a dynamic threshold adaptive detection method is considered. Figure 8 shows the dynamic threshold adaptive detection map of the EDAD model; when we give a threshold of 0.5, we can clearly detect that there are 8 pulse signals.

4.3. Experiment 3: Recovery Experiment of Weak Pulse Signal

It is known from the results of Experiment 1 that there is a weak signal other than in , so the weak signal in the observed signal can be estimated. It is assumed that 4000 data of the observation signal contain pulse signals of different amplitudes at 8 times. The sum of squared errors of EDAD model is 0.018904, which is less than the threshold . Therefore, the DEDR model can be used to recover the weak pulse signal based on the EDAD model. The experimental results are shown in Table 2 and Figure 9, where .

The results in Table 2 show that the error of the predicted value and the true value is low, which is basically below 0.002%. This indicates that the DEDR model has better performance in recovering weak pulse signals in the background of chaotic noise, and the recovered weak pulse signal has higher precision.

Figure 9 shows the detection and recovery results of weak pulse signal. For the convenience of observation, the predicted value is shifted to the right by 70 units, and SIR reaches −85.21699 dB. It can be seen from the results of Table 2 and Figure 9 that the fitting effect between the true value and the predicted value is very good.

4.4. Experiment 4: Detection and Recovery Experiments of Pulse Signals of Different Intensities

Assuming that the period of the weak pulse signal is constant, the intensity of is controlled by changing the magnitude of , that is, , . Similarly, 4,000 points of the observed signal are selected as samples, and the signal is detected by the EDAD model and recovered by the DEDR model. The experimental results are shown in Table 3:

It can be seen from the results shown in Table 3 that the ability of the DEDR model to recover signals also changes with the gradual change of the pulse signal intensity. When the SIR is greater than −85.21699 dB, the values of NMSE and MSE are relatively small, and the effect of restoring weak signals is ideal. However, when the SIR is less than −85.21699 dB, the value of NMSE is significantly increased. At this time, the ability of the DEDR model to recover weak signals is extremely weak. This is because the pulse signal is too strong to destroy the geometry of the chaotic noise background signal, and if the pulse signal is too weak, it will be blurred by some components of the chaotic noise background signal. Therefore, when the SIR is higher than −85.21699 dB, the DEDR model has a better effect of recovering the weak signal. At this time, both the NMSE and MSE are relatively small, and it can be seen that the DEDR model has a low SIR threshold for detecting weak signals.

4.5. Experiment 5: Performance Comparison of Different Models

The proposed model is compared with the LM model, SVM model, and RBF neural network model (RBF) [32, 33]. Select and use the complex autocorrelation method and Cao’s method to determine the delay time of and embedding dimension . Use the EDAD model for signal detection and the DEDR model for signal recovery and use SIR and root mean square error RMSE (), Acc, and AUC to judge the effect of the models. The results are shown in Figure 10 and Table 4:

As can be seen from Figure 10 and Table 4, RMSE is 0.000170, Acc is 1, AUC is 1, and SIR is −85.21699 dB. RMSE value is on the order of 10e − 4, which is higher than RMSE value obtained by other models, and SIR value is much lower than the SIR values of other models. Specific results are shown in Table 4. These are sufficient to show that the DEDR model has a stronger detection ability and is estimated to perform better.

5. Conclusions

Using the short-term predictability of chaotic time series and sensitivity to small disturbances, combined with phase space reconstruction, the Elman deep learning adaptive detection model (EDAD) and double-layer Elman deep learning neural network recovery model (DEDR) are constructed. The model built in this paper does not need to know prior knowledge of dynamic equations of the chaotic system and pulse signal. Weak pulse signal in the background of chaotic noise can be detected under the condition that nonlinear mapping is unknown. It is a simple and easy to understand and apply model for detecting and recovering weak signals. From the experimental results, the following conclusions can be drawn. The EDAD model can effectively detect weak pulse signals from the background of chaotic noise, and Acc, AUC, and DTAD can effectively and objectively judge whether signals are detected. Weak pulse signal estimated by the DEDR model has high precision, and the error between the predicted value and true value is basically below 0.002%. It can be seen from the experiments of different intensity pulse signal detection and recovery that the model constructed in this paper can achieve lower signal-to-interference ratio working threshold and keep the prediction accuracy at a higher level in the background of chaotic noise with less data. MSE can be as low as 0.018904 when SIR is as low as −85.21699 dB; it is intuitive to show the model constructed in this paper has stronger detection ability and better estimation effect. In the next step, we will continue to improve the impact of prediction error of the EDAD model on the construction of the detection and recovery model, in order to achieve a lower signal-to-interference ratio threshold and to extend the detection and recovery of other weak signals.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This study was supported by the Fundamental and Advanced Research Project of CQ CSTC of China (grant no. cstc2018jcyjAX0464), National Natural Science Foundation of China (grant no. 11871124), and Chongqing University of Technology Foundation (grant no. 2016ZD33).