Mathematical Problems in Engineering

Mathematical Problems in Engineering / 2021 / Article

Research Article | Open Access

Volume 2021 |Article ID 5570945 | https://doi.org/10.1155/2021/5570945

Tuan Anh Pham, Van Quan Tran, Huong-Lan Thi Vu, "Evolution of Deep Neural Network Architecture Using Particle Swarm Optimization to Improve the Performance in Determining the Friction Angle of Soil", Mathematical Problems in Engineering, vol. 2021, Article ID 5570945, 17 pages, 2021. https://doi.org/10.1155/2021/5570945

Evolution of Deep Neural Network Architecture Using Particle Swarm Optimization to Improve the Performance in Determining the Friction Angle of Soil

Academic Editor: Venkatesan Rajinikanth
Received19 Feb 2021
Accepted23 Apr 2021
Published07 May 2021

Abstract

This study focuses on the use of deep neural network (DNN) to predict the soil friction angle, one of the crucial parameters in geotechnical design. Besides, particle swarm optimization (PSO) algorithm was used to improve the performance of DNN by selecting the best structural DNN parameters, namely, the optimal numbers of hidden layers and neurons in each hidden layer. For this aim, a database containing 245 laboratory tests collected from a project in Ho Chi Minh city, Vietnam, was used for the development of the proposed hybrid PSO-DNN model, including seven input factors (soil state, standard penetration test value, unit weight of soil, void ratio, thickness of soil layer, top elevation of soil layer, and bottom elevation of soil layer) and the friction angle was considered as the target. The data set was divided into three parts, namely, the training, validation, and testing sets for the construction, validation, and testing phases of the model. Various quality assessment criteria, namely, the coefficient of determination (R2), mean absolute error (MAE), and root mean square error (RMSE), were used to estimate the performance of PSO-DNN models. The PSO algorithm showed a remarkable ability to find out an optimal DNN architecture for the prediction process. The results showed that the PSO-DNN model using 10 hidden layers outperformed the DNN model, in which the average correlation improvement increased R2 by 1.83%, MAE by 5.94%, and RMSE by 8.58%. Besides, a global sensitivity analysis technique was used to detect the most important inputs, and it showed that, among the seven input variables, the elevation of top and bottom of soil played an important role in predicting the friction angle of soil.

1. Introduction

In geotechnical engineering, the internal friction angle of soil is one of the most important parameters [1]. This index is mentioned in many studies in the field of geology, for instance, in the investigation of the foundation bearing capacity [27], soil stability [3, 810], limit loads [11], and slope stability [9, 1214]. In slope stability or retaining walls-related problems, the friction angle was proven to affect the soil slope stability significantly, as an increase of the friction angle decreases the soil consolidation [3]. Similarly, the retaining wall’s reaction is also significantly affected by the internal friction angle [8]. In another study, You Mo et al. [11] pointed out a positive correlation between friction angle and critical load. The above points showed that the friction angle is a vital soil parameter [1], and more in-depth research of the latter is required. However, the friction angle is only mentioned indirectly in related geological studies [24, 8, 9, 11], and there are limited researches on this index.

The value of the internal friction angle depends on many soil parameters, such as grain size, particle shape, unit weight, specific gravity, and soil moisture [15, 16]. Two main approaches were proposed for estimating the friction angle of soil: experimental and theoretical approaches. Concerning the experimental one, the internal friction angle can be obtained from the direct shear test [1, 1719] and the triaxial shear test [4, 1922]. However, these experiment testing methods possessed some disadvantages, as they are time-consuming and required relatively expensive equipment [2325]. Besides, theoretical and numerical methods were also applied to determine the friction angle of soil. As an example, Mo et al. [11] proposed an equation for the friction angle of the soil and confirmed the validity of the theoretical method.

Notwithstanding the foregoing, the theoretical method’s limitation is that it is necessary to be based on some assumptions, in this case, the ideal rigid-plastic slip-line field theory. Other studies proposed alternative methods to evaluate the internal friction angle of soil, based on the standard penetration test (SPT) values [2629] and cone penetration test (CPT) results [30, 31]. Those methods provided several empirical formulas that could quickly predict the soil friction angle. However, these formulas contained only one or several main input variables, which could reduce the prediction accuracy in many cases.

In the last two decades, based on the development of computer science, numerous studies have proposed a novel approach for surveying and assessing geological issues, namely, the artificial intelligence (AI) approach [10, 25, 32, 33]. In Mikaeil et al.’s work [32], the authors have used the genetic algorithm (GA) to study three crucial physical and mechanical characteristics of soil, in which the internal friction angle is an important parameter. Additionally, an adaptive neuro-fuzzy inference system (ANFIS) has been used by Murlidhar et al. [25] to study the shear strength of rock based on the internal friction angle. Das and Basudhar [34] and Al-Hamed et al. [35] have developed ANN models to predict the internal friction angle of soil. Besides, Pham et al. [36] have developed a hybrid model using random forest and particle swarm optimization for estimation of undrained shear strength of soil. The obtained results showed the effectiveness and reliability of AI approach, reflected by the high R2 values achieved [10, 25, 32, 33]. Besides, Deep Neural Network (DNN) is also another powerful and efficient algorithm of AI. Nowadays, DNN has become popular and widely used to solve practical engineering problems and provide reliable results [3740]. The problem with such an ML approach is that it has a large number of critical hyperparameters, making it difficult to find an optimal model architecture. Among various optimization algorithms, Particle Swarm Optimization (PSO) is an efficient, robust, and straightforward algorithm mainly used to solve problems that are difficult to find an exact mathematical model [4143].

Therefore, the main objective of this study is to develop a PSO-DNN hybrid model, which can be self-developed to find the best architectural model to predict the internal friction angle of the soil, including the number of hidden layers and number of neurons in each hidden layer. Seven input factors that might affect the prediction of friction angle of soil were considered: the soil state, standard penetration test value, unit weight of soil, void ratio, thickness of soil layer, top elevation of the soil layer, and bottom elevation of the soil layer. For this aim, a data set consisting of 245 soil samples was collected from some drill holes in Ho Chi Minh City, Vietnam. The database was then divided into the training (60% of the data set), validation (20% of the data set), and testing (20% of the data set), related to the training, validation, and testing phases of the hybrid model. In addition, a global sensitivity analysis method using Monte Carlo simulation was conducted to find out the most important parameter that affects the prediction of friction angle of soil.

2. Data Collection and Preparation

2.1. Experimental Measurement of Friction Angle of Soil

In this work, the authors conducted 245 identified different soil samples in Ho Chi Minh City, Vietnam (Figure 1), and the results are summarized in Table 1. To determine the characteristics of the soil properties, we use the method of determination of shear laboratory resistance in a shear box apparatus [44, 45]. The test samples with the original structure and natural moisture were prepared by cutting from the original soil samples into blocks, which were taken into the ring knives by the method of determining the volume by the ring knife [46]. The underside and top of the soil sample should be leveled with the circular knife's edge and placed with dampened paper first. For fast cutting without draining, the absorbent paper must be replaced with tracing paper (or thin plastic) [46]. Simultaneously with cutting soil samples, it is necessary to take soil to determine moisture content.


SN30GeHZ1Z2(ϕ)
Unit(kN/m3)(m)(m)(m)(o)

13818.950.8463.6004.3007.90016.983
24819.330.7823.4001.1004.50020.042
341719.470.76514.0001.20015.20022.328
43818.90.8423.1003.2006.30015.600
.........
.........
2433518.680.9306.6002.0008.6008.870
24461620.630.5605.4004.0009.40016.420
24564420.410.58413.10040.00053.10016.902

Min2216.20.4160.8000.0002.0004.23
Average4.64914.62919.240.7846.30513.36919.67316.648
Max98521.571.39433.00077.32079.32033.4
SD1.88210.9400.9480.1945.56014.41415.5906.953

With sandy soil, the sample was prepared by pouring dry sand onto a cutter installed in a hard bottom box with many small holes. Then, take a sample into a cylindrical ring knife with two ends, place the sample in the cutting box, and compress the sample under pressure σ. If soil does not preconsolidate, cut samples immediately, cut quickly, and cut samples at a speed of 1 mm/minute until the sample is damaged. If the soil consolidates under pressure σ, maintain that compressive force until the level of consolidation is met, and then proceed as normal. It is required to cut slowly at a cutting rate of 0.01 mm/min (or slower) until the sample breaks. The destructive force is the maximum value read on the strain gauges. The vertical compression pressure was applied at four levels: 50 kPa, 100 kPa, 200 kPa, and 300 kPa, respectively. For each level of compressive pressure, the corresponding shear strength of soil was recorded. Draw the shear strength according to the vertical pressure, thereby determining the cohesion of soil C, tgϕ and calculating the internal friction angle (ϕ).

2.2. Data Preparation

To predict the internal friction angle of the soil, the inputs need to be carefully considered. First, the water content was not included in this study, as it is believed that the effect was taken into account in the number of SPT blows [47]. Besides, many studies have been successful in finding the relationship between the results of in situ experiments (CPT and SPT) and the Plastic and Liquid Limited humidity [4851]. Since the SPT is one of the most commonly used tests in practice to indicate the soil's ability to withstand compression and in situ shear, it is believed to relate to the factors that characterize the ability of the shear strength of the soil. Many studies have shown the relationship between the in situ experiment results and the internal friction angle of the soil [2631]. Second, this study aims to provide a tool to quickly analyze the internal friction angle of the soil based on basic parameters that can be quickly and simply determined instead of parameters that take time and cost for determination. Therefore, this study selected SPT value (N30) of soil as the main feature, and some other indicators such as soil state (S), unit weight of soil (G), void ratio (e), thickness of soil layer (H), top elevation of soil layer (Z1), and bottom elevation of soil layer (Z2) are inputs to predict the internal friction angle of soil. The internal friction angle of soil (ϕ) is the single output variable.

One of the most popular methods to determine the internal friction angle of soil (ϕ) is the direct shear test method [44, 45]. There are three soil direct cutting modes: UU mode (this mode is fast cutting-no cohesion, no drainage), CU mode (fast cutting, cohesive-no drainage), and CD mode (slow cutting, consolidation-drainage; this mode usually applies to sandy soils) [1]. Firstly, the UU cutting mode is suitable for fast construction and difficult drainage, and the soil samples in the present study were all under this type of condition. Secondly, the UU cutting mode is quick test, taking under 30 minutes to perform, while the CU and CD modes take longer to execute and can take weeks or even months to complete. Besides, the shear strength parameters given by the UU cutting mode are generally safer than those in the other two modes. Therefore, in this study, the results of the UU cutting mode were applied to determine the friction angle of all soil samples. The meaning of the soil type state number is explained in Table 2. Besides, the parameters’ meanings are illustrated in Figure 2, and the histograms of all the input and output variables are shown in Figure 3.


Soil typeSoil state numberSoil type-state

Clay1Very soft clay
2Soft clay
3Firm clay
4Stiff clay
5Very stiff clay
6Hard clay

Sand7Loose sandy
8Medium sandy
9Dense sandy

The data set containing 245 samples was statistically introduced and summarized in Table 1, including several soil samples, min, max, average, and standard deviation of the input and output variables. It can be seen that, among 245 soil samples, 46 samples are sandy soil and the rest are clayey soil. Soil state number (S) ranged from 2 to 9, corresponding to soft clay to dense sandy soil. The minimum depth of the soil samples is 2.0 m, and the maximum depth is 79.32 m. The standard penetration test values (N30) were from 2 to 85, where 2 corresponds to the soft clay soil and 85 corresponds to the hard clay soil. The unit weight of soil (G) ranged from 16.2 (kN/m3) to 21.57 (kN/m3) in accordance with the clay soil to sandy soil layers. The void ratio (e) varied from 0.416 to 2.194, corresponding from medium sandy soil to soft clay soil. The internal friction angles of soil ranged from 4.23 (degrees) to 33.4 (degrees), related to the shear strength of the soil increasing from soft clay to medium and dense sandy soil. These input data results are consistent with the properties of the soil layers. An example of 50 data samples is given in Table 3.


N°SN30GeHZ1Z2ϕ
UnitkN/m3Mmm(degree)

13818.950.8463.64.37.916.983
24819.330.7823.41.14.520.042
341719.470.765141.215.222.328
43818.900.8423.13.26.315.6
571018.820.7904.3711.328.18
63819.700.68531.14.117
73819.700.68531.14.117
84919.440.7963.10.53.618.73
93718.610.8874.73.78.414.85
103319.100.8062.312.71515.295
113319.100.8062.312.71515.295
1285021.280.4311111233.4
132417.801.1930.83.248.02
142417.801.1930.83.248.02
1541519.180.7813.510.51418.2
162217.101.2721.53.95.47.03
1782420.120.5439.525.935.432.8
1841320.100.58526.48.63524.17
1941320.100.58526.48.63524.17
2081520.900.4576.19.415.532.57
213418.980.8294.526.514.533
223418.980.8294.526.514.533
2382419.700.6902.728.831.529.3
244919.440.7525.30.7618.1
2582220.200.5716.933.14031.03
2641920.200.56430.156.736.8524.2
2782620.210.5736.433.64030.855
2882620.210.5736.433.64030.855
293617.781.07015.54.52013.854
3081220.900.4605556031.75
3165020.550.5715.668.47425.2
3281420.670.4775.73338.731.623
3352019.380.7445.831.537.320.744
3481819.520.6743.452730.4529.08
353718.840.8799.31.210.514.572
3681220.700.50047.211.230.725
373418.850.85141.55.513.742
383418.850.85141.55.513.742
393718.940.8457.7714.714.4
403718.940.8457.7714.714.4
4134219.400.714524.629.615
4251819.670.7056.70.3720.883
4351819.670.7056.70.3720.883
4431019.200.792210.512.513.87
4531019.200.792210.512.513.87
4641418.300.8305.42.47.816.433
4781320.260.57013.46.62031.205
4842219.430.7471.21011.216.766
4941418.850.7922.25.8816.315
5081219.150.7353576027.505

In this study, we divided the data into three data sets, in which the training set was used to build the model, the validation set was used to give an estimate of model skill while tuning the model’s hyperparameters, and the testing set was used to estimate of the skill of the final tuned models to choose the best model. The test set was fixed at 20% of the total data set, hidden during training, and used only to evaluate the performance of the final model after the hyperparameters tuning process.

3. Machine Learning Methods

3.1. Particle Swarm Optimization

Particle Swarm Optimization (PSO) was presented by Kennedy and Eberhart [52]. It became prevalent because it is a continuous optimization process type and allows an analysis of multiple targets. For a continuous search of the best solution, the method used is to move the positions of particles at a given velocity calculated in each iteration. Each movement of particles is influenced by its best position and the best position in the entire search space (Figure 4). This is expected to move the swarm to the best position. The effectiveness of the solution is assessed through a fitness function. In addition, PSO does not use the slope of the problem to be optimized, which means that PSO does not require optimization issues to be differentiable like standard optimization methods such as gradient descent and quasi-newton methods. PSO is a powerful technique that has been widely used for optimization issues in many fields in general and geotechnical engineering in particular [53, 54].

The pseudocode of the algorithm is presented as follows. (Algorithm 1)

FOR each particle i in swarm
 Initialize parameters: , _damp, c1, c2.
FOR each dimension j
  Initialize position xij randomly within permissible range
  Initialize velocity randomly within permissible range
End FOR
End FOR
Iteration k = 1
WHILE k < maximum_Iteration
 FOR each particle i in swarm
  Calculate fitness value
  IF fitness value > P_best[i] THEN
   P_best[i] = fitness value
  END IF
  IF fitness value > G_best THEN
   G_best = fitness value
END IF
END FOR
FOR each particle i in swarm
FOR each dimension j
   Calculate new velocity:
(k + 1) = wvij(k) + c1random (0,1) (P_best[i] − xij[i]) + c2random (0,1) (G_best − xij[i])
   Update particle positon:
   xij(k + 1) = xij(k) + (k + 1)
END FOR
END FOR
 = ._damp
k = k + 1
END WHILE

The parameters in the equation that defines the velocity of the next iteration are as follows: is an inertial parameter; c₁ and c₂ are the acceleration coefficients; c₁ value gives the importance of individual best solution, and c2 is the importance of global best candidate solution; _damp is the inertial reduction coefficient, and it helps the swarm movement to quickly converge.

3.2. Deep Neural Network

Deep neural network (DNN) is definitely one of the most advanced regression methods. DNN model is analogous to a multistage regression. The main idea is to create a flexible nonlinear statistical model consisting of several layers and neurons in each layer. In DNN, each node in a layer is associated with a certain weight, denoted as ij, with every node in the other layers creating a fully linked neural system [55]. Except for the input layer, each node is a neuron that uses a nonlinear activation function [56]. Thanks to multilayer and nonlinear activation functions, the DNN model could distinguish nonlinear separable data. The DNN structure used in this study is a multilayer perception (MLP), shown in Figure 5. Further details of the DNN model can be found in [57].

3.3. Particle Swarm Optimization-Deep Neural Network Algorithm

The PSO-DNN hybrid model was created based on the PSO algorithm, in which the fitness function of each particle in the swarm was DNN. The hybrid algorithm is implemented with the number of hidden layers ranging from 2 to 10. For each case of the number of hidden layers, random particles are generated, the length of the particle vector is the number of hidden layers, and the jth dimension of the particle vector is the number of hidden neurons. The swarm architecture was presented in Figure 6, in which k is the number of hidden layers and N is the number of particles in a swarm. The flow chart of the hybrid PSO-DNN model was shown in Figure 7. In this algorithm, the number of hidden layers in the model is incremented from 2 to 10. For each case of the number of hidden layers, the algorithm will automatically look for the optimal model architecture for that number of hidden layers.

3.4. Performance Evaluation

To evaluate the effectiveness of the proposed AI models, three widely used statistical criteria are applied, namely, mean absolute error (MAE), root mean square error (RMSE), and the coefficient of determination (R2). In order to estimate the percentage change of data which can be achieved by prediction, it is common to use the R2 coefficient in regression analysis [58]. To measure the average intensity of error, both MAE and RMSE were used [59]. Studies have concluded that the model will be more accurate when R2 is closer to 1 and MAE and RMSE is closer to 0. These coefficients are determined by the following formulas:where represents the number of the samples, and were the actual and predicted outputs, respectively, and was the average value of .

4. Results and Discussion

4.1. Effect of Training Data Size

Because the testing set size was selected and fixed to 20% of the total data set and hidden from the training process, the training data set size was chosen by dropping from 60% to 20% of the data set, or meaning the size of the validation set increased from 20% to 60% of the remaining data set. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. The initialization parameters of the DNN model used in this section are presented in Table 4.


ParametersValue and description

Number of hidden layers2
Neurons structure(25, 25)
Use early stoppingTrue
Activation functionTanh
Cost functionMSE
Data usedTraining/validation/testing data set
Number of runs (shuffle input data)300

From Figure 8, it can be seen that training performance progressively increased with decreasing in the training set. The average R2 value was increased from 0.952 to 0.991 when the training set size was decreased from 60% to 20%. On the other side, the predictions showed less accurate results on the validation set and testing set as the training set size decreases. The average R2 value was decreased from 0.889 to 0.602 on the validation set, while the average R2 value dropped from 0.919 to 0.570 on the testing set. In addition, the standard deviation from the DNN models was increased with decreasing in training set size. To be more specific, the standard deviation was increased from 0.0277 to 0.351 on the testing set. The above results show that the performance of the model got better and became more stable with the most significant training set size case. Aiming to increase the prediction accuracy of the model, the training set size of 60% was selected in this study.

4.2. DNN Structure Optimization

The evolutionary results of PSO-DNN models are evaluated in this section. The initialization parameters of PSO-DNN used in this study are given in Table 5. Figure 9 illustrates the evolution of the PSO-DNN model through 30 generations with the number of hidden layers set to values: 2, 4, 6, 8, and 10. A summary of the best predictability of the models is presented in Table 6.


ParametersExplanationValue and description

N_swarmSwarm size50
max_iterationNumber of iterations30
iInitial weight0.5
_dampInitial weight damping0.9
C1Personal learning coefficient0.4
C2Global learning coefficient0.8
VarMinNumber of min neurons2
VarMaxNumber of max neurons100
N_hiddensNumber of hidden layers2, 4, 6, 8, 10
VmaxVelocity max0.05  (VarMax − VarMin)
VminVelocity minVmax
Fitness functionDNN
Activation functionTanh
Early stoppingHandling overfittingTrue
Cost functionR2, MAE, RMSE
Data usedTraining/validation dataset


CriteriaNumber of hidden layersBest criteria valueRankBest neuron structure

R220.9304(28, 32)
40.9262(20, 36, 35, 29)
60.9275(17, 44, 52, 52, 38, 28)
80.9313(15, 54, 77, 37, 15, 78, 12, 19)
100.9381(8, 49, 90, 32, 19, 77, 36, 22, 89, 29)

MAE (o)21.4654(9, 19)
41.4262(24, 53, 62, 21)
61.4443(9, 76, 27, 13, 56, 77)
81.5285(32, 25, 51, 74, 36, 35, 22, 76)
101.3951(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)

RMSE (o)21.9255(7, 22)
41.9554(23, 36, 34, 41)
61.9213(9, 61, 31, 23, 41, 70)
81.8391(17, 55, 47, 54, 66, 39, 41, 28)
101.8712(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)

Bold values represent the best model for each performance indicator.

It can be seen that the DNN model with 10 hidden layers has evolved to bring about better performance; the best generation yielded a correlation of R2 = 0.938, MAE = 1.395, and RMSE = 1.846. The model with the next best performance is the DNN model with 8 hidden layers which produce accurate intermediate precision (R2 = 0.931, MAE = 1.528, and RMSE = 1.839). The DNN model with 10 hidden layer gives the best performance for R2 and MAE criteria, while the model with 8 hidden layers gives the best result for the RMSE cost function. However, the best neuron structure for each cost function is different and is also shown in Table 6. The evolutions of number of neurons in 10 hidden layers and 8 hidden layers of the best DNN model referring to R2, MAE, and RMSE are presented in Figure 10. It means that the evolution of the model has found three good results; choosing the best model will be done in the next section.

4.3. Predictive Capability of the Models

In this section, the three best models, 10 hidden layers, R2 (10-HD-R2), 10 hidden layers, MAE (10-HD-MAE), and 8 hidden layers, RMSE (8-HD-RMSE) were chosen for comparison with each other. From the point of view of statistical probability, the random factors in the input variables should be considered. Therefore, 300 simulations take into account the randomness of the input variable sequence number. The data used in this section include training, validation, and testing data sets. The criteria for selecting the best model are based on the results of the testing data set. The results are shown in Figure 11 and Tables 79 .


ModelData setAverageMinMaxSD

10-HD-R2Training0.9460.9150.9700.0083
Validation0.8950.7210.9390.0377
Testing0.9350.8610.9660.0138

10-HD-MAETraining0.9000.5860.9530.0477
Validation0.8510.4360.9300.0657
Testing0.8890.5490.9390.0453

8-HD-RMSETraining0.9350.8260.950.0156
Validation0.9010.6220.9420.0343
Testing0.8810.7950.9270.0186


ModelData setAverage (o)Min (o)Max (o)SD

10-HD-R2Training1.1880.8831.4230.0899
Validation1.7211.3922.5090.2345
Testing1.3401.0361.7150.1339

10-HD-MAETraining1.6361.0502.5600.3031
Validation2.1051.4733.4550.3411
Testing1.8231.2432.8700.271

8-HD-RMSETraining1.2681.1012.1070.1534
Validation1.7061.2673.4440.2564
Testing1.5231.2382.3380.1923


ModelData setAverage (o)Min (o)Max (o)SD

10-HD-R2Training1.5661.2601.8480.1088
Validation2.2741.7823.4980.3395
Testing1.771.4062.5920.1835

10-HD-MAETraining2.0591.4213.1280.3711
Validation2.6681.8765.0950.473
Testing2.2551.5783.3100.2988

8-HD-RMSETraining1.7221.4653.5980.2129
Validation2.221.6764.8670.3861
Testing2.0191.6133.8820.3264

The results showed that the predicted results on the training and validation sets gave good accuracy with satisfactory average accuracy R2 ranging from 0.851 to 0.946, MAE ranging from 1.188 to 2.105, and RMSE varying from 1.566 to 2.668, in which it can be seen that, among three good models, the 10-HD-R2 gives the best prediction, which yielded an average correlation of R2 = 0.935, MAE = 1.34, and RMSE = 1.77 on the testing data set. The next best model is 8-HD-RMSE, which produces accurate intermediate precision (R2 = 0.881, MAE = 1.523, and RMSE = 2.019). Moreover, the standard deviation of the 10-HD-R2 model is better than that of the 8-HD-RMSE, where SD = 0.0138, 0.1339, and 0.1835 for R2, MAE, and RMSE criteria compared to SD = 0.0186, 0.1923, and 0.3264 of two models, respectively.

Table 10 shows the comparison between the DNN model and the PSO-DNN model. The DNN model received initialized parameters as shown in Table 11. The best 10-HD-R2 PSO-DNN model was included for comparison with the proof of the effectiveness of the algorithm. The average error criteria of 300 simulations were compared to confirm the model’s performance. It can be seen that the 10-HD-R2 PSO-DNN model outperformed the DNN model, which improved the average correlation of R2 by 1.83%, MAE by 5.94%, and RMSE by 8.58%.


CriteriaDNN modelPSO-DNN modelPerformance improvement (%)

R20.9180.9351.83
MAE (o)1.4251.3405.94
RMSE (o)1.9361.7708.58


AuthorModelParameterNumber of samplesRR2RMSE (o)MAE (o)

Khanlari et al. [60]ANNInternal friction angle2000.890.7922.261.92
MVR0.890.792.3851.929
Mohammadi et al. [61]MLPInternal friction angle1080.9020.8149.2856.137
MLR0.7190.5175.0823.954
Iyeke et al. [62]ANNInternal friction angle830.8970.8054.774.34
Present studyDNNInternal friction angle2450.9580.9181.9361.425
PSO-DNN0.9670.9351.771.34

Table 11 presents some research results on ML applications in determining the internal friction angle of soil. The results of the present study, as well as other studies, show the effectiveness of the ML technique in predicting the internal friction angle of the soil with expected R2 from 0.79 to 0.935 on the testing data set. However, due to the different data sets used and the input variables, it makes no sense to compare these results. A project using different data sets as well as various input variables is needed to provide a general model for the prediction of soil shear parameters.

Figure 12 shows a visual comparison of test results and predictions based on the friction angle of soil of the DNN and PSO-DNN models. The performances of two ML models have been tested on all three data sets: training, validation, and testing.

4.4. Sensitivity Analysis

In this section, a global sensitivity analysis was conducted to evaluate the importance of input parameters for the model using Monte Carlo methods [63]. It is an effective way to investigate a relationship between input and output. The input data set taken from Saltelli’s sampling scheme was used to develop the DNN model [64]. The global sensitivity index is determined by the following formulas:where Var(Y) denotes the total variance of the model output; d denotes the number of input features; Vari denotes the model output variance in response to variation of the ith input variable; Varij denotes the model output variance in response to the simultaneous variation of the ith and the jth input; STi denotes the total sensitivity index.

The result of the total sensitivity analysis was shown in Figure 13. It can be seen that, among the 7 input variables used to predict the friction angle of soil, the elevation of the soil layer bottom (Z2) was the most important feature, which achieved an average sensitivity index score of 0.625. The topsoil layer elevation (Z1) was the second important variable, confirmed by an average sensitivity index score of 0.603. From the point of view of soil mechanics, the deeper the soil is, the more compact it is, so the depth of the soil sample plays an important role in detecting the friction angle of the soil when its density, state, and many other mechanical properties change according to the depth. The variables H, G, and S were ranked as the third to the fifth important predictors, with an average sensitivity index ranging from 0.203 to 0.05. Other predictor variables included in the model (e, G) had lower than 0.01 in the sensitivity index, indicating that they had not affected the output prediction. It is important to note that the sensitivity analysis in this section is only relevant for the input data set itself and cannot be confirmed with other data sets.

5. Conclusions

In this study, a PSO-DNN hybrid model, which can evolve itself to find out the best models, was developed to predict the friction angle of soil. The hybrid model evolved on its own and found three of the best among the survey models. A database containing 245 soil samples from geological boreholes was used to develop and evaluate the three proposed DNN models: 10-HD-R2, 10-HD-MAE, and 8-HD-RMSE.

The results show that the models’ performances improved and stabilized from R2 = 0.602 to 0.889 on the validation set with the increase in training data set size from 20% to 60%. Research results show that PSO-DLNN obtains the best results with the number of hidden layers from 8 to 10. The optimal number of neurons in each hidden layer is not the same and is distributed intricately in the hidden layer. It recommends that a DLNN model with 8 to 10 hidden layers might be optimal for the problem related to predicting the friction angle of soil. However, it is advisable to select the number of neurons in each hidden layer through evolutionary methods to bring about high efficiency to the DLNN model. The results also showed that, on the training and validation data sets, all three best models, 10-HD-R2, 10-HD-MAE, and 8-HD-RMSE, have good predict results, in which the leading is the 10-HD-R2 model on the testing data set; the 10-HD-R2-DNN model still gave the best results and outperformed the other two models. Prediction results of 300 simulations show that the 10-HD-R2-DNN model has a smaller standard deviation, indicating that the model is more stable than the other two models.

In addition, the sensitivity analysis using the Monte Carlo method was carried out to evaluate the importance of input features in the model study. The results show that the two inputs related to the depth of the soil layers (Z2 and Z1) were considered to be the most important parameters for predicting soil friction angle.

Data Availability

The data used in the manuscript are available in the supplementary materials. Resharing of data with other researchers is completely allowed; however, they need to be cited in the researches.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Supplementary Materials

Data collected in Ho Chi Minh City, Vietnam, including 245 experimental results for determining the internal friction angle of the soil, together with the input parameters required for machine learning modeling. (Supplementary Materials)

References

  1. M. Ghazavi, M. Hosseini, and M. Mollanouri, “A comparison between angle of repose and friction angle of sand,” 2008. View at: Google Scholar
  2. A. Namdar and G. S. Gopalakrishna, “Analysis of mixed soil model characteristics,” Modern Applied Science, vol. 3, no. 5, 2009. View at: Publisher Site | Google Scholar
  3. A. S. Abedi and N. Hataf, “Bearing capacity of strip footings located on grouted soil slopes,” Electronic Journal of Geotechnical Engineering, vol. 19, no. Z4, pp. 16827–16837, 2014. View at: Google Scholar
  4. T. Schanz and P. A. Vermeer, “Angles of friction and dilatancy of sand,” Géotechnique, vol. 46, no. 1, pp. 145–151, 1996. View at: Publisher Site | Google Scholar
  5. G. G. Meyerhof, “Some recent research on the bearing capacity of foundations,” Canadian Geotechnical Journal, vol. 1, no. 1, pp. 16–26, 1963. View at: Publisher Site | Google Scholar
  6. A. S. Vesic, “Analysis of ultimate loads of shallow foundations,” Journal of the Soil Mechanics and Foundations Division, vol. 99, no. 1, pp. 45–73, 1973. View at: Google Scholar
  7. A. Namdar and M. K. Pelko, “Numerical analysis of soil bearing capacity by changing soil characteristics,” Frattura Ed Integrità Strutturale, vol. 3, no. 10, pp. 38–42, 2009. View at: Publisher Site | Google Scholar
  8. K. S. M. SEYED and A. R. RASTI, “Evaluation of soil-structure interaction parameters in static and dynamic response of the retaining wall,” 2017. View at: Google Scholar
  9. P. Jiang and M. S. Zhang, “Real-time monitor method of soil slope stability in seasonal frozen area,” Applied Mechanics and Materials, vol. 303–306, pp. 777–781, 2013. View at: Publisher Site | Google Scholar
  10. K. Gelisli, T. Kaya, and A. E. Babacan, “Assessing the factor of safety using an artificial neural network: case studies on landslides in Giresun, Turkey,” Environmental Earth Sciences, vol. 73, no. 12, pp. 8639–8646, 2015. View at: Publisher Site | Google Scholar
  11. Y. Mo, X. C. Liu, Z. H. Guo, B. Teng, Z. Y. Zou, and Z. Y. Zhu, “Slip-line field theory's application in soil subgrade,” Applied Mechanics and Materials, vol. 170–173, pp. 283–288, 2012. View at: Publisher Site | Google Scholar
  12. H. Yang, J. Wang, and Y. Liu, “A new approach for the slope stability analysis,” Mechanics Research Communications, vol. 28, no. 6, pp. 653–669, 2001. View at: Publisher Site | Google Scholar
  13. X. Zhou, J. Li, Z. Liu, and Y. Tang, “Analysis of slope stability with imprecise soil properties using uncertain sets,” Mathematical Problems in Engineering, vol. 13, 2019. View at: Google Scholar
  14. X. Chu, L. Li, and Y. Cheng, ““Risk assessment of slope failure using assumption of maximum area of sliding mass and factor of safety equal to unit,” Advances in Civil Engineering, vol. 23, 2019. View at: Google Scholar
  15. S. Altun, B. Göktepe, and A. Sezer, “Relationships between shape characteristics and shear strength of sands,” Soils and Foundations, vol. 51, no. 5, pp. 857–871, 2011. View at: Publisher Site | Google Scholar
  16. P. Vangla and G. M. Latha, “Influence of particle size on the friction and interfacial shear strength of sands of similar morphology,” International Journal of Geosynthetics and Ground Engineering, vol. 1, no. 1, p. 6, 2015. View at: Publisher Site | Google Scholar
  17. N. Marachi, J. Duncan, C. Chan, and H. Seed, “Plane-strain testing of sand,” Laboratory Shear Strength of Soil, vol. 23, 1981. View at: Publisher Site | Google Scholar
  18. T. Schanz and J. Alabdullah, “Testing unsaturated soil for plane strain conditions: a new double wall biaxial device,” Springer Proceedings in Physics, vol. 23, pp. 169–178, 2016. View at: Publisher Site | Google Scholar
  19. K. L. Dev, R. J. Pillai, and R. G. Robinson, “Drained angle of internal friction from direct shear and triaxial compression tests,” International Journal of Geotechnical Engineering, vol. 10, no. 3, pp. 283–287, 2016. View at: Publisher Site | Google Scholar
  20. A. L. Kryzhanovskii, Y. S. Vil'gel'm, and S. V. Medvedev, “Determination of the friction angle of soils in triaxial-compression apparatus and shear devices,” Soil Mechanics and Foundation Engineering, vol. 22, no. 3, pp. 105–111, 1985. View at: Publisher Site | Google Scholar
  21. Ş. Cioară, A. Stanciu, and I. Lungu, “Comparative study ON determining the internal friction angle for sand,” Soil Mechanics and Foundation Engineering, vol. 12, 2014. View at: Google Scholar
  22. S. Nanda and N. R. Patra, “Determination of soil properties for plane strain condition from the triaxial tests results,” International Journal for Numerical and Analytical Methods in Geomechanics, vol. 39, no. 9, pp. 1014–1026, 2015. View at: Publisher Site | Google Scholar
  23. J. C. Santamarina and G. C. Cho, “Determination of critical state parameters in sandy soils - simple procedure,” Geotechnical Testing Journal, vol. 24, no. 2, 2001. View at: Google Scholar
  24. J. H. Choi, S. Dai, J. S. Lin, and Y. Seol, “Multistage triaxial tests on laboratory‐formed methane hydrate‐bearing sediments,” Journal of Geophysical Research: Solid Earth, vol. 123, no. 5, pp. 3347–3357, 2018. View at: Publisher Site | Google Scholar
  25. B. R. Murlidhar, M. Ahmed, D. Mavaluru, A. F. Siddiqi, and E. T. Mohamad, “Prediction of rock interlocking by developing two hybrid models based on GA and fuzzy system,” Engineering with Computers, vol. 35, no. 4, pp. 1419–1430, 2019. View at: Publisher Site | Google Scholar
  26. H. Mujtaba, K. Farooq, N. Sivakugan, and B. M. Das, “Evaluation of relative density and friction angle based on SPT-N values,” KSCE Journal of Civil Engineering, vol. 22, no. 2, pp. 572–581, 2018. View at: Publisher Site | Google Scholar
  27. H. Hettiarachchi and T. Brown, “Use of SPT blow counts to estimate shear strength properties of soils: energy balance approach,” Journal of Geotechnical and Geoenvironmental Engineering, vol. 135, no. 6, pp. 830–834, 2009. View at: Publisher Site | Google Scholar
  28. M. Hatanaka and A. Uchida, “Empirical correlation between penetration resistance and internal friction angle of sandy soils,” Soils and Foundations, vol. 36, no. 4, pp. 1–9, 1996. View at: Publisher Site | Google Scholar
  29. R. Salgado, P. Bandini, and A. Karim, “Shear strength and stiffness of silty sand,” Journal of Geotechnical and Geoenvironmental Engineering, vol. 126, no. 5, pp. 451–462, 2000. View at: Publisher Site | Google Scholar
  30. P. K. Robertson and R. G. Campanella, “Interpretation of cone penetration tests. Part I: Sand,” Canadian Geotechnical Journal, vol. 20, no. 4, pp. 718–733, 1983. View at: Publisher Site | Google Scholar
  31. Y. Yoshida and K. Kokusho, “A proposal on application of penetration tests on gravelly soils,” Denryoku Chuo Kenkyusho Hokoku, vol. 1, no. 1–4, 1988. View at: Google Scholar
  32. R. M.-M. Y. Rad2-Sina, S. Haghshenas, M. Z. Naghadehi, and R. Branch, “Risk management of a mechanized tunneling project through assessment of geological units using an evolutionary method,” 1988. View at: Google Scholar
  33. X. Liangfu and C. Jiangtao, “Predicting the mechanical parameters of holocene alluvial layers in the yangtze river delta using BP neural network based on cone penetration test,” 1988. View at: Google Scholar
  34. S. K. Das and P. K. Basudhar, “Prediction of residual friction angle of clays using artificial neural network,” Engineering Geology, vol. 100, no. 3-4, pp. 142–145, 2008. View at: Publisher Site | Google Scholar
  35. S. A. Al-Hamed, M. F. Wahby, and A. M. Aboukarima, “Artificial neural network for soil cohesion and soil internal friction angle prediction from soil physical properties data,” 2014. View at: Google Scholar
  36. B. T. Pham, C. Qi, L. S. Ho et al., “A novel hybrid soft computing model using random forest and particle swarm optimization for estimation of undrained shear strength of soil,” Sustainability, vol. 12, no. 6, p. 2218, 2020. View at: Publisher Site | Google Scholar
  37. D. T. Bui, P. Tsangaratos, V.-T. Nguyen, N. V. Liem, and P. T. Trinh, “Comparing the prediction performance of a Deep Learning Neural Network model with conventional machine learning models in landslide susceptibility assessment,” Catena, vol. 188, 2020. View at: Publisher Site | Google Scholar
  38. D. T. Bui, “A novel deep learning neural network approach for predicting flash flood susceptibility: a case study at a high frequency tropical storm area,” Science of the Total Environment, vol. 701, p. 134413, 2020. View at: Google Scholar
  39. C. D. Storie and C. J. Henry, “Deep learning neural networks for land use land cover mapping,” 2018. View at: Google Scholar
  40. R. Zemouri, N. Omri, F. Fnaiech, N. Zerhouni, and N. Fnaiech, “A new growing pruning deep learning neural network algorithm (GP-DLNN),” 2019. View at: Google Scholar
  41. F. Vandenbergh and A. Engelbrecht, “A study of particle swarm optimization particle trajectories,” Information Sciences, vol. 176, no. 8, pp. 937–971, 2006. View at: Publisher Site | Google Scholar
  42. M. Khajehzadeh, M. R. Taha, A. El-Shafie, and M. Eslami, “Modified particle swarm optimization for optimum design of spread footing and retaining wall,” Journal of Zhejiang University-Science A, vol. 12, no. 6, pp. 415–427, 2011. View at: Publisher Site | Google Scholar
  43. J. Meier, W. Schaedler, L. Borgatti, A. Corsini, and T. Schanz, “Inverse parameter identification technique using PSO algorithm applied to geotechnical modeling,” Journal of Artificial Evolution and Applications, vol. 2008, 2008. View at: Publisher Site | Google Scholar
  44. P. V. Lade and C.-T. Liu, “Experimental study of drained creep behavior of sand,” Journal of Engineering Mechanics, vol. 124, no. 8, pp. 912–920, 1998. View at: Publisher Site | Google Scholar
  45. R. Saito, H. Fukuoka, and K. Sassa, “Experimental study on the rate effect on the shear strength,” 1998. View at: Google Scholar
  46. T. Aydogmus and H. Klapperich, “Three-dimensional numerical parametric analysis of soil-geosynthetic-interaction,” Journal of Soil Mechanics & Foundations Divisions, vol. 171, p. 8, 2008. View at: Google Scholar
  47. G. G. Meyerhof, “Shallow foundations,” Journal of Soil Mechanics & Foundations Divisions, vol. 91, 2009. View at: Google Scholar
  48. B. Sharma and A. Sridharan, “Liquid and plastic limits of clays by cone method,” International Journal of Geo-Engineering, vol. 9, no. 1, p. 22, 2018. View at: Publisher Site | Google Scholar
  49. S. P. K. Kodicherla and D. K. Nandyala, “Use of CPT and DCP based correlations in characterization of subgrade of a highway in Southern Ethiopia Region,” International Journal of Geo-Engineering, vol. 7, no. 1, p. 11, 2016. View at: Publisher Site | Google Scholar
  50. N. Q. A. M. Yusof and H. Zabidi, “Reliability of using standard penetration test (SPT) in predicting properties of soil,” Journal of Physics: Conference Series, vol. 1082, 2018. View at: Publisher Site | Google Scholar
  51. M. Farzi and A. Saligheh Zadeh, “Determination of relationship between standard penetration test and geotechnical parameters in Ahwaz soil,” Geomechanics and Geoengineering, vol. 23, pp. 1–13, 2019. View at: Publisher Site | Google Scholar
  52. J. Kennedy and R. Eberhart, “Particle swarm optimization,” Proceedings of ICNN’95-International Conference on Neural Networks, vol. 4, pp. 1942–1948, 1995. View at: Publisher Site | Google Scholar
  53. D. Tien Bui, V.-H. Nhu, and N.-D. Hoang, “Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and Multi-layer Perceptron Neural Network,” Advanced Engineering Informatics, vol. 38, pp. 593–604, 2018. View at: Publisher Site | Google Scholar
  54. Y. M. Cheng, L. Li, S.-C. Chi, and W. B. Wei, “Particle swarm optimization algorithm for the location of the critical non-circular failure surface in two-dimensional slope stability analysis,” Computers and Geotechnics, vol. 34, no. 2, pp. 92–103, 2007. View at: Publisher Site | Google Scholar
  55. B. G. Lipták, Process Control Instrument Engineer, Elsevier, London, UK, 1995.
  56. C. Van Der Malsburg and F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Elsevier, Berlin, Heidelberg, 1986.
  57. T. A. Pham, V. Q. Tran, H.-L. T. Vu, and H.-B. Ly, “Design deep neural network architecture using a genetic algorithm for estimation of pile bearing capacity,” PLoS One, vol. 15, no. 12, 2020. View at: Publisher Site | Google Scholar
  58. S. Menard, “Coefficients of determination for multiple logistic regression analysis,” The American Statistician, vol. 54, no. 1, pp. 17–24, 2000. View at: Publisher Site | Google Scholar
  59. T. Chai and R. R. Draxler, “Root mean square error (RMSE) or mean absolute error (MAE)?-arguments against avoiding RMSE in the literature,” Geoscientific Model Development, vol. 7, no. 3, pp. 1247–1250, 2014. View at: Publisher Site | Google Scholar
  60. G. R. Khanlari, M. Heidari, A. A. Momeni, and Y. Abdilor, “Prediction of shear strength parameters of soils using artificial neural networks and multivariate regression methods,” Engineering Geology, vol. 131-132, pp. 11–18, 2012. View at: Publisher Site | Google Scholar
  61. M. Mohammadi, S. M. Fatemi Aghda, M. Talkhablou, and A. Cheshomi, “Prediction of the shear strength parameters from easily-available soil properties by means of multivariate regression and artificial neural network methods,” Geomechanics and Geoengineering, vol. 0, no. 1, pp. 1–13, 2020. View at: Publisher Site | Google Scholar
  62. S. Iyeke, E. Eze, J. Ehiorobo, and S. Osuji, “Estimation OF shear strength parameters OF lateritic soils using artificial neural network,” Nigerian Journal of Technology, vol. 35, no. 2, p. 260, 2016. View at: Publisher Site | Google Scholar
  63. I. M. Sobol, “Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates,” Mathematics and Computers in Simulation, vol. 55, no. 1, pp. 271–280, 2001. View at: Publisher Site | &publication_year=2001" target="_blank">Google Scholar
  64. A. Saltelli, “Making best use of model evaluations to compute sensitivity indices,” Computer Physics Communications, vol. 145, no. 2, pp. 280–297, 2002. View at: Publisher Site | Google Scholar

Copyright © 2021 Tuan Anh Pham et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Related articles

No related content is available yet for this article.
 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views397
Downloads326
Citations

Related articles

No related content is available yet for this article.

Article of the Year Award: Outstanding research contributions of 2021, as selected by our Chief Editors. Read the winning articles.