Abstract
Monitoring and prediction of ground settlement during tunnel construction are of great significance to ensure the safe and reliable operation of urban tunnel systems. Data-driven techniques combining artificial intelligence (AI) and sensor networks are popular methods in the field, which have several advantages, including high prediction accuracy, efficiency, and low cost. Deep learning, as one of the advanced techniques in AI, is demanded for the tunnel settlement forecasting problem. However, deep neural networks often require a large amount of training data. Due to the tunnel construction, the available training data samples are limited, and the data are univariate (i.e., containing only the settlement data). In response to the above problems, this research proposes a deep learning model that only requires limited number of training data for short-period prediction of the tunnel surface settlement. In the proposed complete ensemble empirical mode decomposition with adaptive noise long short term memory (CEEMDAN-LSTM model), single-dimensional data is divided into multidimensional data by CEEMDAN through the complete ensemble empirical mode decomposition. Each component is then predicted by a LSTM neural network and superimposed for obtaining the final prediction result. Experimental results show that, compared with existing machine learning techniques and algorithms, this deep learning method has higher prediction accuracy and acceptable computational efficiency. In the case of small samples, this method can significantly improve the accuracy of time series forecasting.
1. Introduction
The gradual ground settlement in the tunnel construction process may result in serious damages and disasters, such as water inrush and mud outburst, rock burst, rock and soil deformation, displacement and slump, gas and other harmful gas outbursts, high temperature and heat damage, freezing and thawing, and acid groundwater corrosion [1]. Accurate prediction of tunnel surface settlement is crucial for construction companies to prevent such disasters [2]. With the fast development of artificial intelligence (AI) technology and Internet of Things (IoT) techniques, it is highly demanded to estimate and forecast the ground settlement in the near future with the data collected by sensor networks [3, 4].
Many scientists have conducted a lot of research on the settlement prediction of tunnels with sensor data. The research methods can be roughly divided into the theoretical calculation empirical method and sensor-based data measurement and analysis method. The theoretical calculation empirical method is represented by the Peck empirical formula method, including the numerical analysis method, numerical simulation method, semitheoretical analysis method, and random theoretical model [5, 6]. Sensor-based data measurement and analysis methods are divided into statistical methods and machine learning methods. In recent years, with the rapid development of computing technology, artificial neural networks (ANN), and fuzzy comprehensive evaluation (FCE), wavelet analysis and support vector machines (SVM) have been widely used in short-term time series forecasting [7]. In 2006, Hinton et al. proposed the concept of the deep neural network (DNN) [8]. Since then, deep learning (DL) has gradually become the most popular method in the field of machine learning. The long short-term memory (LSTM) network extends the traditional recurrent neural networks (RNN) and is widely adopted for the sensor-based time series data forecasting problems [9]. The LSTM network has many successful application cases in the fields of pattern recognition, machine translation, audio and video processing analysis, traffic flow prediction, medicine, etc., which can be easily adopted for the tunnel surface settlement problem [10–14].
There are two main difficulties in tunnel settlement prediction. First, the length of historical data of tunnel construction is limited. Second, the data collected is usually univariate. In the research of Hu et al., through a series of comparative studies, three machine learning techniques were selected, namely, backpropagation neural network, extreme learning machine, and support vector regression machine, as the best way of short-term predictions of tunnel settlement under various conditions [15]. Based on this research and using the same dataset, this research shows that there is a complex nonlinear relationship between tunnel settlement and many random uncertain factors; it is difficult to predict tunnel settlement using a single machine learning technique.
This research proposes a new deep learning theoretical framework for tunnel settlement time series prediction and builds the complete ensemble empirical mode decomposition with the adaptive noise long short term memory (CEEMDAN-LSTM) model on this basis. In order to verify the performance of the proposed method, a real tunnel surface settlement from a tunnel construction company was selected. The purpose of this research is to find the most suitable data-driven prediction technology for tunnel surface settlement. This research has the following contributions.(1)The complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) is used to decompose the original univariate short-period tunnel settlement data into multidimensional data, thereby increasing the dimension of the data.(2)In this study, combining the feature extraction ability of CEEMDAN and the prediction ability of LSTM, the prediction accuracy of tunnel settlement was improved.(3)Explain the improvement of prediction performance through comprehensive comparative research. In the experimental part, the result of comparing the proposed method with the existing work is explained. First, we prove that the CEEMDAN-LSTM model is better than the traditional machine learning model. Next, more experiments were performed to compare the proposed model with traditional machine learning methods. Through a series of comparative studies, the performance of the proposed deep learning model is proved.
2. Related Works
Short-term time series data forecasting is a hot topic in the fields of regression analysis, machine learning, and intelligent computing. Generally speaking, there are two methods for time series data forecasting, namely, model-based methods and data-driven methods. Model-based methods use the mathematics of the physical model to simulate and usually require recording of multivariate data. The accuracy of the prediction depends on the validity of the physical assumptions. Chakeri et al. [16] explored the influence of important attributes on tunnel settlement through experience. Among all the methods, the most important factor in finding ground settlement is the diameter of the tunnel. Loganathan and Poulos [17] predicted the ground motion of the tunnel around the clay. For the tunnel in the homogeneous clay, the observed and predicted settlement and horizontal motion are consistent. Zhao [18] used recognized analytical methods to calculate the lining force and deformation of shallow tunnels and deep tunnels. Strokova [19] investigated settlement prediction methods based on traditional models during tunnel construction. Ercelebi et al. [6] used the Plaxis finite element program to predict the settlement by the finite element method, and the results showed that the finite element model can predict the short-term surface settlement well under a given volume loss value. In short, the model-based method provides a white box model for the tunnel settlement problem. At the same time, multiple external variables can be used under valid mathematical assumptions, and the prediction accuracy of the model-based method is comparable to that of the data-driven method.
The data-driven method is a gray box or black box model, which involves complex internal structures, accepts input datasets and preprocessed versions, and outputs comprehensive prediction results. Zhang [20] comprehensively compared the prediction performance of five commonly used machine learning algorithms in tunnel induced settlement combined with the meta-heuristic algorithm. The experimental results show that BPNN has good extrapolation ability. Yan [21] proposed a new integrated machine learning model Adaboost.RT framework. Experimental results show that compared with existing machine learning techniques and algorithms, the integrated learning has higher and acceptable computational efficiency. Chen [22] used three artificial neural network (ANN) methods to predict the maximum surface settlement caused by EPB shield tunnel construction. Experimental results show that the GRNN model is superior to BP and RBF neural networks in terms of accuracy and calculation time. Li et al. [23] introduced a probability principal component analysis method estimating missing data in the structure health monitoring system.
Empirical mode decomposition is a new adaptive signal time-frequency processing method proposed by Huang et al. in 1998, which is especially suitable for the analysis and processing of nonlinear and nonstationary signals [24]. Wang et al. [25] compared the application of EMD and EEMD in seismic time-frequency analysis. Experimental results show that, compared with EMD, EEMD solves the problem of mode mixing, and the time-frequency spectrum obtained more truly reflects the real geological conditions. Bedi and Toshniwal [26] proposed a deep learning method based on empirical mode decomposition (EMD), which combines the empirical mode decomposition method with the long- and short-term memory network model to estimate the time of a given season, day, and day interval power requirements. Zhang [27] proposed a CEEMDAN-CLSFPA combined model for short-term wind speed prediction. The experimental results show that the CEEMDAN combined model can take advantage of the advantages of a single model and has the best performance in the single model and the benchmark model. Cao [28] combined CEEMDAN with the LSTM neural network for financial time series data forecasting. Li et al. [29] proposed a hybrid LSTM model coupling empirical mode decomposition for missing data prediction in structural health monitoring (SHM) systems.
3. Methodology
3.1. EMD
In order to solve the difficulty of artificial neural network processing random data, the original nonlinear and nonstationary tunnel surface settlement time series S (t) are preprocessed. Empirical mode decomposition is an adaptive signal time-frequency processing method proposed by Huang et al. It decomposes the signal according to the time-scale characteristics of the data itself and does not need to set any basis functions in advance. It decomposes the series into a finite number of intrinsic modal functions (IMFs). There are many factors that cause tunnel settlement, and these factors represent different characteristics of the original signal on different time scales. The EMD decomposition process is called the screening process, and the steps are as follows: (1) find all the maxima and minima in the time series . (2) All the extremum points are fitted to the upper envelope and the lower envelope by the cubic spline interpolation function. (3) Calculate the mean of the upper envelope and the lower envelope by . (4) Subtract from the original series to give a new series . (5) Repeat steps (1) through (4) with as a new input series until the mean of approaches zero, and get the IMF, denoted , as its index. (6) is separated from the original series to get a difference series without high frequency components . (7) The steps (1) to (6) are repeated with as the new input series until the termination condition is satisfied (typically such that the last residue satisfies monotonicity). Through the above steps, we screened a series of IMFs, recorded as . The original signal is reconstructed by these IMFs:where is the residue, representing the trend of the time series.
3.2. CEEMDAN
The EMD has great advantages in dealing with nonstationary and nonlinear signals, but still has the mixing mode problem. The mixing mode problem refers to the presence of extremely similar oscillations in different modes or very disparate amplitude in a mode. By adding Gaussian white noise to the signal, the ensemble empirical mode decomposition (EEMD) algorithm largely eliminates the mixing mode problem in the EMD algorithm [30]. However, the EEMD algorithm cannot completely eliminate Gaussian white noise after signal reconstruction, which causes reconstruction errors. To solve these problems, the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) was proposed as an improved version of EEMD [31]. It eliminates mode mixing more effectively, the reconstruction error is almost zero, and the cost of calculation is greatly reduced. Define the operator that produces the mode obtained by EMD, and let be white noise with normal distribution N (0, 1). The steps of the CEEMDAN algorithm are the following: (1) decompose each by EMD to extract the first IMF, where is a noise coefficient, , and define the first mode as . (2) Calculate the first residue . (3) Decompose residue to obtain the second mode as . (4) Repeat for another IMF until the obtained residue. The final residue can be expressed aswhere M is the total number of IMFs. The IMFs together compose the characteristics of the original signal at different timescales. The residue clearly shows the trend of the original sequence, which is smoother and reduces the prediction error effectively.
Figure 1 shows the data waveform of the time series of the tunnel settlement monitoring points in the actual project after CEEMDAN decomposition, and the waveforms are arranged from high frequency to low frequency.
3.3. LSTM
The one-step prediction of the tunnel settlement time series requires not only the latest data but also the previous data. Benefiting from the self-feedback mechanism of the hidden layer, the RNN model has advantages in dealing with long-term dependencies, but it has difficulties in practical applications [32]. In order to solve the problem of the disappearance of the RNN gradient, Sepp Hockert and Jürgen Schmidhuber proposed the LSTM model in 1997, which was recently improved and popularized by Alex Graves [9, 33]. Compared with traditional prediction algorithms, LSTM fully considers time memory and conforms to the characteristics of time series data. The LSTM unit consists of a storage unit that stores information, and the information is updated by three special gates: input gate, forget gate, and output gate. The LSTM unit structure is shown in Figure 2.
Time is t, is the input data of the LSTM unit, is the output of the LSTM unit at the previous moment, is the value of the storage unit, and is the output of the LSTM unit. The calculation process of the LSTM unit can be divided into the following steps.(1)First, calculate the value of the candidate storage unit , is the weight matrix, and is the deviation:(2)Calculate the value of the input gate , and the input gate controls the update of the current input data to the state value of the storage unit, is the sigmoid function, is the weight matrix, and is the deviation:(3)Calculate the value of the forget gate , and the forget gate controls the update of the historical data to the state value of the storage unit, is the weight matrix, and is the deviation:(4)Calculate the value of the memory unit at the current moment, and is the state value of the last LSTM unit: Among them, “” stands for the dot product. The update of the storage unit depends on the state value of the last unit and the candidate unit and is controlled by the input gate and the forget gate.(5)Calculate the value of the output gate , and the output gate controls the output of the state value of the storage unit, is the weight matrix, and is the deviation:(6)Finally, the output of the LSTM unit is calculated:
Benefitting from three control gates and storage units, LSTM can easily save, read, reset, and update long-term information. It is worth noting that due to the sharing mechanism of the internal parameters of the LSTM, the dimensionality of the output can be controlled by setting the dimensionality of the weight matrix. LSTM establishes a long-time delay between input and feedback. The gradient will neither explode nor disappear because the internal state of the memory cell in this structure maintains a continuous flow of errors. The tuned hyperparameters of the LSTM neural network are shown in Table 1.
3.4. Proposed Method
Based on the theoretical framework, a deep learning hybrid prediction model, namely, the CEEMDAN-LSTM model, is constructed, which extracts depth and abstract features and applies it to short-term tunnel settlement prediction. The deep learning hybrid prediction model combines empirical mode decomposition and its variants with LSTM. Figure 3 details the flow chart of the framework, including three stages:(1)First, use EMD and CEEMDAN to decompose the original tunnel settlement time series into several series and a residual component .(2)Several sequences and residual components will be obtained as the input data of the LSTM prediction model for training, and the prediction results will be obtained. The prediction results of the test set are and .(3)Finally, the prediction results obtained from each IMF and remaining components are reconstructed according to the following formula to obtain the final prediction time series.where is the length of the test sequence, is the prediction sequence of each , is the prediction sequence of the remaining components, and is the final prediction sequence of the test set.
The iterative loss curve of each IMF is shown in Figure 4. From Figure 4, it is noted that the training and validation losses converge without overfitting.
4. Experimental Results
Applying the CEEMDAN-LSTM model to the actual tunnel settlement prediction, two datasets were collected by the local China Tunnel Construction Company. One measures the surface settlement of the tunnel constructed by Ningbo Metro Line 3 in Ningbo city, China. The other dataset measures the surface settlement of the subway constructed in Zhuhai, China. The original data was available at https://downloads.hindawi.com/journals/mpe/2019/7057612.f1.zip, which was uploaded by Hu [15]. Among the 1000+ surface points measured in the two tunnel projects in China, we randomly selected tunnel surface points Nos. 180, 184, 186, 189, and 205 for the Zhuhai subway and Nos. 552, 554, 559, 569, and 570 for the Ningbo Metro Line 3 for experimental results’ demonstration. The results of the rest surface points are similar. Readers may refer to our source code at http://github.com/Cy743652/CEEMDAN-LSTM for further verification. Since the sample data is a one-dimensional time series, one problem is that the sample dimension is too low. It is necessary to select the appropriate size of the rolling window for expanding the original single-dimensional data into multidimensional data. According to the scale and experience of the dataset, the optimal rolling window size is between 1 and 6. In the actual experiment, the size of the rolling window is finally at 2. Then, the original one-dimensional sample data is expanded into multidimensional sample data, thereby solving the problem of too low dimensionality. In addition, for each tunnel ground point, 4/5 records are used as the training dataset. The remaining 1/5 of the records are verified as test data.
The surface settlement record data of the No. 186 measurement point of the Zhuhai Rail Transit Metro Tunnel and the No. 552 measurement point of the construction of Ningbo Metro Line 3 are selected for simulation. Figure 5 shows the prediction results of the two measurement points. It can be seen that, in both datasets, the proposed model has achieved good prediction results.
(a)
(b)
In order to further prove the effectiveness of the proposed model, some existing time series forecasting models are used for comparison. The compared models include traditional machine learning models, such as support vector regression (SVR), BP neural network, extreme learning machine (ELM), and LSTM model, combining LSTM with empirical mode decomposition (EMD) or a complete set with adaptive noise Hybrid model of empirical mode decomposition (CEEMDAN). Figures 6–10 show the prediction results of ground point numbers 180, 184, 186, 189, and 205 of the Zhuhai Rail Transit Metro Tunnel, Zhuhai city, China. The test results are enlarged in the form of subgraphs.
Figure 11 shows the prediction of tunnel settlement at point 554 in the construction of Ningbo Metro Line 3. Due to underground human interference, most of the measurements were carried out during the construction process and declined later. The movement trend of the surface points in the first stage is useless for predicting the test time period. Figures 12–15 show the prediction results of surface point numbers 552, 559, 569, and 570 of Ningbo Metro Line 3 in Ningbo city, China. The experimental results show that this method can predict well the movement of the tunnel surface points subject to human disturbance.
In order to evaluate the predictive performance of the model and verify the effectiveness of the proposed method, the experiment uses 3 evaluation indicators. Evaluation indicators include average absolute error (MAE), root mean square error (RMSE), and average absolute percentage error (MAPE). The calculation formula of each evaluation index is
In the three evaluation metrics formulas, is the actual value of the ground sensor point, is the predicted value, is the average of the true values, and is the total number of samples.
Use MAE, RMSE, and MAPE to evaluate the error level of the prediction results. The smaller the value, the more accurate the prediction effect.
For all the measurement points shown above, the MAE, RMSE, and MAPE results of SVR, ELM, BP, and LSTM are listed in Table 2. The experimental results show that compared with the three machine learning techniques, LSTM does not achieve better forecast results. Because LSTM is a deep learning model, the experimental data set of tunnel settlement is not large enough to make it difficult to saturate the training of the LSTM model. Therefore, a single LSTM model is not suitable for tunnel settlement prediction.
Table 3 lists the MAE, RMSE, and MAPE results of BPNN, LSTM, EMD-LSTM, and CEEMDAN-LSTM. The experimental results show that the CEEMAN-LSTM model can predict tunnel settlement more accurately. All the RMSE values are less than 0.65, and the MAPE value is less than 2.5%, indicating that the proposed CEEMAN-LSTM model can be well applied to the actual tunnel settlement prediction problem.
Figures 16 and 17 show box plots of the absolute prediction errors between the predicted and true values of the surface points of tunnels 180 and 554. The upper and lower two short solid lines in the figure represent the extreme absolute prediction errors (0%–100%) of each model. The red line in the middle represents 50% of the absolute prediction error of the model. The bottom of the box plot represents an absolute error of 25%, and the top and bottom represent an absolute error of 75%. Compared with other models, the box plot distribution of CEEMDAN-LSTM is the smallest, indicating that the model has the highest prediction accuracy and the smallest absolute prediction error.
5. Conclusion
Aiming at the nonlinear, nonperiodic, and nonstationary complex characteristics of tunnel settlement time series data, a hybrid neural network framework combining the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) with multiple LSTM neural networks is proposed to accurately predict the settlements. The proposed method inherits the multiple LSTM neural network deep learning structure that existed in the literature, such as [34–36].
While the CEEMDAN was used to decompose the original tunnel settlement time series data into IMFs series according to frequency, multiple LSTM neural networks were adopted to construct training and prediction models for each IMF component. The results of the prediction subsequences are combined to obtain the final prediction result.
Compared with existing similar time series forecasting models, the CEEMDAN-LSTM method combines the forward and backward data characteristics of the LSTM neural network to output accurate forecast results. Compared with EMD decomposition, CEEMDAN effectively avoids modal mixing and improves the prediction accuracy of the model.
This paper uses the settlement monitoring data of two actual tunnel projects in China for verification. Using traditional machine learning methods, LSTMs model, and a hybrid model combining LSTMs and decomposition methods, a comprehensive comparative study was carried out to verify the prediction quality of the proposed model. Based on the experimental results, the proposed model has the superior performance compared to all existing models in terms of prediction error and trend prediction accuracy. We are confident that the model is suitable for real-world tunnel construction works for tunnel surface settlement estimation and forecasting.
Data Availability
The data used in this article is confidential and only available internally for the contruction company.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
K.Y. conceptualized, provided methodology, and validated the study. Y.C. provided the software, performed formal analysis, and visualized the study. X.Z. investigated, provided resources, and curated data. Y.C. and K.Y. wrote and prepared the original draft. Y.C. and K.Y. wrote the review and edited the draft. K.Y. and X.Z. supervised the study. K.Y. and X.Z. did project administration. K.Y and X.Z. acquired funding for the study.
Acknowledgments
This work was supported by Zhejiang Provincial Natural Science Foundation of China under Grant no. LY19F020016.