Abstract
As one of the core equipment of aircraft, avionics provide a power source for flight. Avionics are complex and highly susceptible to environmental factors. Failure in the long flight process is also relatively large, affecting the stability and safety of aircraft operation. Therefore, it is of great significance to predict the typical faults of avionics. At present, a lot of research achievements have been made on the fault diagnosis of avionics, but the failure prediction of avionics is rarely involved, especially the middle and long-term fault prediction. Hence, this paper proposes a fault prediction method for avionics based on an echo state network. In particular, one-dimensional wavelet denoising filtering and z-score standardized preprocessing are carried out to obtain pure useable data first. Then, the set training data are input into the ESN model. When the model is well trained, the test data can be used to test the model. Finally, the experimental results demonstrate that the proposed ESN model can effectively improve the medium and long-term prediction accuracies of the faults in avionics equipment. Besides, the proposed model can not only identify the types of faults but also predict the specific time when the faults occur. It guarantees the safe and stable operation of the equipment and supports the stable development of the air transport industry, which has great theoretical and practical application value.
1. Introduction
The rapid development of China’s economy has promoted the leaps and bounds development of China’s aviation field. Avionics provide continuous power output for civil and military aircraft. They are the core part of the aircraft and should be used to the maximum extent possible to ensure their normal operation [1, 2]. If a failure occurs, it is bound to cause economic and personnel safety losses, but the occurrence of failure always exists; analysis and research for the typical failure of avionics equipment can predict the failure or anomaly in advance and, to a certain extent, can reduce the loss to the minimum. The failure of avionics not only causes huge economic losses but also may affect national security. So, it is very important to analyze the reliability of avionics. At present, electronic equipment is more and more widely used in aerospace products, facing more and more severe working environments. An important goal of electronic equipment structure design is how to ensure high reliability in order to ensure that electronic equipment is in a vibration and impact environment with high reliability [3]. Modern military aircraft need to perform a variety of complex tasks, and the onboard equipment needs to ensure that it can work in different complex environments, which makes the safety and reliability of the electronic equipment on the aircraft become more and more demanding. As a result, airborne equipment is prone to failure, especially airborne electronic equipment. For example, when the electronic equipment is overworked, the aircraft may break down during the flight, thus affecting the overall performance of the aircraft or even causing serious consequences. However, in actual flight, avionics faced a more complex environment, such as the excitation of its own engine and the disturbance of external pressure, which have a great influence on its reliability [4, 5].
The manufacturing industry and the electronic equipment in the manufacturing industry are also developing in the direction of more intelligence, so the problem of the time prediction of the failure of electronic equipment arises. From the technical difficulty, due to the rapid development of ’China’s manufacturing industry and the automated electronic equipment industry, the demand for the manufacturing industry is growing [6, 7]. In order to meet the demand of the manufacturing industry and other industries in automatic electronic equipment, the internal structure of automatic electronic equipment is gradually becoming more and more complicated, advanced, and intelligent. Suppose there is no special technical personnel to maintain the electronic equipment once the electronic equipment fails. First, it will lead to a longer maintenance cycle of electronic equipment, affecting the normal use of electronic equipment. Second, because of the unnecessary inspection, repeated inspection, and incorrect inspection of electronic equipment, it is easy to cause the under-maintenance and over-maintenance of electronic equipment, which leads to an increase in maintenance costs and waste. From the perspective of cost, on the one hand, as the structure of electronic equipment becomes more and more precise and its functions become more and more intelligent, the price of purchasing electronic equipment will become more expensive. On the other hand, when professional and technical personnel are hired to maintain electronic equipment, the cost will become higher. From the point of view of maintenance, maintenance can be divided into maintenance after failure and periodic maintenance of electronic equipment. For the maintenance after the failure, when the maintenance is not timely, the possible cost loss cannot be measured. When the maintenance is timely, it will also cause the breakdown of the whole electronic equipment due to the removal of parts. For periodic maintenance, although it can avoid the adverse impact of failure, it will carry out a lot of unnecessary periodic maintenance of electronic equipment [8, 9].
In addition to ensuring basic safety issues, the health status of avionics systems needs to be provided in real time, so that faults can be predicted and located accurately and quickly in a short time once they occur. In addition, if the failure can be predicted successfully, it can provide a reference and basis for avionics maintenance and improve the safety and reliability of equipment operation. Therefore, it is extremely urgent to improve the safety and reliability of avionics and reduce the occurrence of major accidents. This is not only an important way for various small and medium-sized enterprises to improve their competitiveness but also an inevitable trend of the entire aerospace system. And because of the increasing intelligence of modern weapons and equipment, the Prognostics and Health Management (PHM) system has been developed in the United States and other countries. It uses sensor information and expert diagnosis, intelligent inference model, and fault prediction algorithm. The maintenance capability and fault prediction capability of aviation system is one of the main research directions of PHM [10, 11].
Through reading relevant literature, we know that in recent years, the prediction of failure time of electronic equipment has become a hot research topic. Often, people do not know when the next device failure will occur, and it takes more labor and resources to find the fault. Therefore, through reasonable technical means to predict and accurately maintain the failure of aerospace electronic equipment, not only the service life of electronic equipment can be prolonged but also the safety requirements of aviation electronic equipment can be met. How to deal with time series is very important in fault prediction technology, such as predicting the time when the fault occurs. Time series data is the trend of some variables changing with time, and processing time series is to find out these trends for predictive analysis [12, 13]. Time series prediction means that for those complex system whose precise mechanism model cannot be established, it takes the experimental or observed multivariate time series as the entry point to study the internal change rules of the system and predict the future changes of the system. Before analyzing the time series, the data are preprocessed, including smoothing, denoising, and removing outliers. In the global prediction model, all observation samples are regarded as research objects, and the dynamic persistence of the unknown system is studied by establishing the corresponding nonlinear mapping relationship.
Traditional global prediction models mainly use quotient polynomials to achieve global approximation. Local prediction models can be divided into linear and nonlinear models according to different model properties. Compared with the global prediction model, the local prediction model has the advantages of a simple mathematical model, fast training speed, no complicated parameter estimation, and wide application. The above global prediction model and local prediction model are established under the condition of relatively complete observation data. If the avionics operation data are missing or have time-varying characteristics, the prediction results will be greatly affected. The adaptive fault prediction model is a kind of separation and demand method which appears in recent years. Because this method can adjust parameters adaptively according to the current observation data, it is suitable for the situation of missing data or insufficient training data. At present, nonlinear adaptive prediction models mainly include two types, one based on series expansion and the other based on nonlinear function transformation. However, these two methods mainly focus on single-step prediction, and their multistep prediction needs further exploration [14, 15].
Based on the above analysis, the purpose of this paper is to develop a medium and long-term prediction model for avionics based on ESN, which requires sufficient accuracy of multistep prediction and as low computational complexity as possible. At the same time, due to the variation of avionics operating parameters, the statistical characteristics of the observed multivariate time series may change with time. Therefore, the model should have the function of updating parameters in real time and tracking the changing trend of faults online.
2. Related Work
Fault prediction is the core content of fault evaluation, which still belongs to fault diagnosis in a broad sense. It is an extended analysis of fault diagnosis and also the symbol of advanced PHM technology. The purpose is to predict the performance degradation trend and the remaining service life of the system components before the failure occurs. Trend prediction is to estimate the current state and future development trend of system components by statistical analysis of features. Residual life prediction can be regarded as an extended analysis of trend prediction. Usually, monitoring data are compared with characteristic historical trajectories so as to estimate the remaining normal service time of the previous system components. At the same time, there are methods based on experience and failure models. At present, the classification of fault prediction technologies has not been unified. By summarizing the research on mainstream fault prediction technologies, it can be divided into three categories, as shown in Figure 1. From the figure, we know that the knowledge-based approach has the widest application range, and the narrowest is the mechanism-based model. And the prediction accuracy and difficulty show the opposite trend.
Fault prediction method based on the mechanism model: This method requires that a mathematical model reflecting the physical law of performance degradation of the research object can be established. By integrating the environmental load measured by sensors with the damage model selected according to the failure mode of the system or component, precise prediction results can be calculated by calculating the component performance degradation caused by the accumulated load. The physical models commonly used to describe system or component failure include the crack propagation model, fatigue spalling propagation model, and so on. However, the failure physical model of the complex system is very complicated, and some physical characteristics are stochastic and complex, which limit the application of this method in practical engineering. Data-driven fault prediction methods [16, 17]: Through machine learning, multivariate statistical analysis, and other methods, the health behavior model of system components can be learned from historical monitoring data, and the future health status trend or remaining service life of the system components can be evaluated by means of trend change, threshold judgment, degradation curve comparison, and other means. The data-driven method is widely used in failure prediction of complex mechanical systems such as avionics because it does not need prior knowledge of system components and obtains key information from sensor historical data, reducing dependence on historical fault data. Fault prediction method based on an expert system: When the component or system lacks sensor monitoring data or the physical model of the system is difficult to establish, but there are enough historical failure data, the method can be used for data analysis, and the life distribution law of the object can be studied by means of fitting. The fault prediction method based on the life distribution model selects suitable life models such as exponential distribution, normal distribution, and Weibull distribution according to different object characteristics and then makes statistical analysis on historical data to determine the parameters of the distribution model. The fault tree analysis-based prediction method counts all the factors that may contribute to the occurrence of faults, establishes the logical block diagram, and realizes the determination and probability estimation of various possible causes of faults by analyzing the number of faults step by step. However, this method is only suitable for failure prediction of a large number of products, not specific to individual prediction and failure causes [18].
Events related to time can also be seen everywhere in real life, such as the click risk of buildings in thunderstorm weather, the number of WeChat steps a person takes every day or the amount of monthly payment, the annual number of business people, and the quarterly price index of a supermarket. Because this paper is based on the data timestamp data to predict the failure time of avionics, it is a typical time series modeling problem. At present, the most commonly used models for chronological modeling are the autoregression (AR) model, moving average (MA) model, and auto regression roving average (ARMA) model. When the time series itself is not stationary series, if its increment is near zero, it can be called stationary series. However, the model has a conditional limit, which not only limits the highest order of the model but also limits the current random disturbance and the past sequence value. Later, relevant researchers started with features and tried to use machine learning methods to model. Machine learning is also popular in recent years [19, 20]. In terms of datasets and machine learning methods, it has improved a lot. It has evolved from simple at the beginning to complex and diversified now. From domestic and foreign scientific research achievements and a large number of experimental results, machine learning has been applied to all walks of life, such as the random forest model, neural network model, logistic regression model, naive Bayes model, and so on, and achieved good results. From the point of view of our country at present, with the enhancement of data acquisition ability, it has gradually evolved from general dataset to massive dataset, and the computing hardware capacity of major Internet factories (namely, computing power) is also becoming more and more powerful. As a result, the model expression ability of the machine learning method gradually becomes stronger, and the learning ability of the dataset also becomes significantly stronger. However, a series of problems affecting prediction accuracy, such as long training time, gradient explosion, and easy over-fitting, should also be considered [21].
The analysis and prediction of the failure trend of avionics equipment can greatly reduce aircraft flight safety accidents and national economic losses. Research institutions and airlines are committed to the improvement and innovation of avionics fault prediction algorithms. To improve the stability and safety of avionics operation, it is imperative to predict and analyze the failure trend of avionics in time. Foreign research in the field of avionics fault prediction technology started early in 1950. The Palo Alto Company of the United States introduced the fault diagnosis device for the first time to maintain an aeroengine. In the 21st century, the corresponding intelligent methods have been paid attention to by many foreign researchers and widely used in various fields. In 2018, the avionics company Rolls-Royce partnered with an American AI company uptake to use artificial intelligence to predict aeroengine problems. Although neural network technology is widely used in fault prediction, many researchers often combine it with other technologies to solve the defects of neural network prediction, so as to achieve a better prediction effect [22]. Although the research in the field of avionics fault prediction started late in China, it has been devoted to the research of fault prediction technology. In the parameter prediction method, the traditional time series prediction method and artificial neural network intelligent method are used most. Domestic scholars often combine neural networks with other technologies to improve the prediction difficulty of electronic equipment such as complex systems. Generally speaking, people compare and screen out the prediction model with the best result from different prediction model methods and exclude the other models in order to improve the prediction accuracy. However, different prediction models are not exclusive; each has its own advantages and disadvantages. In the field of fault prediction, different models learn sequence information from different angles. If different models can be used to combine and complement each other, it is possible to further increase the accuracy of prediction. The mixed forecasting method improves the shortcomings of the single forecasting method, gives full play to the advantages of each method, and also makes up for their shortcomings. The fault prediction of an aeroengine can obtain the fault information of the engine in advance, so that the fault maintenance can be carried out in time, thus reducing the economic loss and improving safety.
The authors of [23] presented the case where the test mode of a single failure is sufficient to cover all multiple failures. Since the signed directed graph model can only make a qualitative analysis of the complex model for the electronic equipment class, the quantification of judgment accuracy is crucial. The introduction of a membership degree in fuzzy theory makes quantitative analysis possible and breaks the concept of either/or, but fuzzy theory ignores randomness when determining the boundary, so it is difficult for fuzzy theory to associate fuzziness with randomness. And the degree of membership is usually given by expert experience, which is inevitably subjective. The authors of the literature [24] first selected appropriate characteristic values for learning by the binary classification method, then inputted the selected key features into the fast model for learning time series information, predicted future time series by using the multicore method, and selected a large number of real stock history information for verification. The feasibility of the proposed time series prediction method in stock price prediction is illustrated. The authors of reference [25] revealed through the relevant data of bearings in the operation process of electronic equipment that the infinite hidden Markov model is applied to carry out health monitoring, and the hidden state of the single-layer model is divided, so as to achieve fault prediction. The ESN model is a promising multistep forward time series prediction strategy that has been used to predict time series data effectively. At present, the avionics system integrates airborne equipment parameter setting and instrument display function into a unique display control system through advanced liquid crystal display, integrated circuit, communication equipment and software technology, forming a centralized control, distributed management structure. Although the ESN model has good medium and long-term prediction performance, it has too many parameters to update and iterate.
As aircraft systems become more and more complicated, various organizations or individuals have developed a lot of analysis software to study the failure of avionics systems. Current avionics fault prediction systems are generally combined with artificial intelligence algorithms. Some excellent fault prediction methods based on the ESN model have appeared in some published journal papers. Although many reliable and efficient algorithms emerge endlessly, these algorithms are still difficult to deal with long-term fault prediction. It is still a long way to improve the performance of the ESN model to improve the fault prediction ability of avionics. Based on the above discussion, the main contributions of this paper are given as follows:(1)This paper is the first time to apply the ESN model to the avionics forecasting field(2)The method in this paper realizes the medium and long-term prediction of avionics. Compared with the short-term prediction, the research in this paper has more theoretical and practical significance.
3. ESN-Based Medium and Long-Term Avionics Fault Prediction
3.1. The ESN Model
The recursive neural network has a rich nonlinear dynamics mechanism, but the training process is mostly based on the gradient descent principle, the solving speed is not ideal, and there are local optimal problems. And the training process of recursive connection weights in the network is related to the output, so it is difficult to ensure the stability of the network. If the network is unstable, it cannot be predicted. Therefore, network stability is expected to be ensured in a relatively simple way. Network parameters may be given in advance before training and remain unchanged during training, or network stability can be ensured in real time through some adjustment mechanisms during training.
As a new type of recursive neural network, the traditional ESN includes input layer, hidden layer, and output layer, as shown in Figure 2. Among them, the hidden layer, also known as the reserve pool, is composed of large-scale nodes, which are sparsely connected randomly. In the ESN, the storage pool is used for information processing and storage. Before network training, network parameters are given in advance, and the input connection weights and the internal connection weights of the reserve pool are randomly initialized without changing. Only the connection weights between the reserve pool and the output layer need to be obtained through training.
At time t, state equation and network output of ESN are as follows:where represents the neuron activation function of the reserve pool. represents the output unit activation function. Therefore, equations (1) and (2) can be rewritten as
Specifically, the ESN training process mainly includes the following steps. For a given learning task, the ESN parameters are properly initialized, including the input, reserve pool, and output node number. After the initialization, the ESN status is updated according to formula (3) under the input driver. To prepare for the subsequent calculation of the output weight, the state of the reserve pool at each moment is collected into a state matrix Q, denoted aswhere is the state of all neurons in the reserve pool at time t. Meanwhile, the expected signal corresponding to the input signal at each moment is denoted as
The goal of the ESN training is to make the actual output O(t) of the network approximate the expected value D(t), i.e.,
Thus, the mean square error between O(t) and D(t) in the training stage is minimized.
The above problems are transformed into solving the least square method problem, namely,
The solution to equation (9) can be found by using the following pseudoinverse algorithm:
However, for high-dimensional states, the pseudoinverse algorithm will produce an inappropriate solution and overfitting phenomenon. The ridge regression training algorithm is used in this paper:
The solution to equation (11) iswhere is the regularization coefficient, and E is the identity matrix with dimension N.
ESN is a new type of the burst neural network which is widely used in nonlinear time series prediction modeling. Only the output weights need to be solved by applying the linear regression method, which greatly reduces the complexity of system modeling. However, in practical application, it is found that for a large-scale reserve pool, the small change of system state variable may lead to a huge change in output weight, resulting in an ill-conditioned solution. At present, the ESN model is not applied to long-term fault prediction of avionics. Based on the above discussion, the ESN model and its application in long-term fault prediction of avionics are shown in Figure 3. It mainly includes the data preprocessing, model training and testing, and fault prediction.
4. Experimental Results and Analysis
4.1. Experimental Data Introduction
In this paper, the data collected by sensors and other devices on real avionics are collected from 12 different avionics devices, and the data collected by each device ranges from thousands to tens of thousands. The avionics collected contain 28 characteristic values: equipment number, time, temperature, voltage, and so on. In the ESN in this paper, if the equipment is of the same type or has similarities, the data can be integrated together as a training set to predict the possible failure time interval of such equipment.
This paper uses Matlab and Python to process avionics data. Python has a variety of functions, not only for software and front-end development but also for powerful functions in mathematical calculation. Python even has a special open-source library for the development of machine learning deep learning, which has been hot in recent years. PyTorch is an open-source learning library for deep learning of machine learning in Python. This module can greatly improve the efficiency of deep network calculation and training, not only improving the accuracy of training in the process of deep network calculation but also providing a guarantee for the construction of a more complex deep network model in the future.
4.2. Experimental Results Analysis
In order to verify the influence of different sample ratios on the ESN algorithm, this paper found in the experiment that the accuracy and stability of the algorithm could be improved if some process data before and after test time points were added as unmarked samples to participate in training. What is uncertain is how many recorded samples (HS), pretest samples (BS), and posttest samples (AS) can improve accuracy. In this paper, the change of accuracy was tested by constantly adjusting the proportion of the three in the unmarked sample. As shown in Figure 4, the X-axis represents the proportion of HS and AS in the unmarked sample, and the Y-axis represents the proportion of HS and BS.
It can be seen that the added test node sample effectively improves the accuracy and can reach the maximum value. This is because the process data before and after the addition of test nodes in the unmarked sample can improve the features of failure prediction during training. Since it is difficult to extract precise fault signal features under complex noise, the training of unlabeled samples is equivalent to feature extraction again, which improves the prediction accuracy.
The influence of the proportion of different training samples on the prediction accuracy of the ESN model should be further demonstrated. It can be seen from Figure 5 that with the increase of the proportion of the training set, the distribution of prediction error is concentrated towards zero, and the box graph is more compact, proving that the prediction accuracy becomes higher. When the training proportion is 80%, the prediction accuracy of the model reaches the highest, but when the proportion of the training set continues to increase (set as 90%), the prediction accuracy of the model shows a trend of decline, possibly due to the phenomenon of overfitting of the model.
In addition to the prediction accuracy, the running time of the model is also an important factor to be considered. Figure 6 gives the prediction time of different methods (LSTM (long short-term memory), BP, GRU (gated recurrent unit), ESN, CNNs (convolutional neural networks), and GCN (graph convolutional network)) with increasing sample size. As can be seen from the figure, CNN and GCN models have the longest running time due to their deep network structure. Although they may have high predictive accuracy, their long training time is one of the major drawbacks. In contrast, the running time of the remaining four models will be much reduced. Among them, although the running time of the ESN model proposed in this paper is not the lowest, it is completely acceptable for the current computer level. It can be seen from the above results that the method proposed in this paper not only has high prediction accuracy but also has a relatively short running time, so the comprehensive performance of the model is quite good.
In order to eliminate the influence of data selection on comparison results, the ESN model is used to process the training and test data divided in the previous paper in the same way, and the test degradation curve based on ESN is obtained. It can be clearly observed from Figure 7 that as the operation cycle of test data increases, the smaller the fault prediction interval becomes, the more concentrated it becomes. From the results, its prediction accuracy will be higher, and it is reasonable to establish a confidence interval with 90% test data. It shows that the proposed method can well track the fault development process of avionics and has good performance in fault prediction.
In order to further demonstrate the medium and long-term performance of the proposed method, Figure 8 shows the failure prediction results of the ESN model for avionics in one year. It can be seen from the figure that the MSN value of the model is relatively large during the five months from January to May, mainly because the model did not fit the process data well at the beginning, so the prediction accuracy is not very good. With the increase of time and the accumulation of data, the predicted values of the proposed ESN model can track the real fault data well from June to January of next year and obtain a low MSE value, which indicates that the proposed method has a good performance of medium and long-term fault prediction.
5. Conclusions
With the rapid development of the modern aviation industry, the degree of automation and intelligence of avionics has been significantly improved. The normal operation of avionics depends on the close cooperation between various systems. However, the failure performance of avionics is also very complicated due to the complex and bad aviation environment and heavy equipment workload. In view of the equipment with abnormal operation status, the failure prediction is carried out by using the operation data of the avionics system.
In this context and in view of the existing research on the failure of avionics equipment for the long-term prediction of the problem, specifically, the preprocessed data are input into the model for training and testing, and the ideal medium and long-term fault prediction results are finally obtained. Finally, experimental results verify the superiority and effectiveness of the proposed method. Although the method in this paper has achieved a good prediction effect, ESN is still a shallow model. When encountering big data, the improved deep learning model will be worth studying.
Data Availability
The datasets used during the current study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.