Abstract
This research carries out a comparative study to investigate a machine learning solution that employs the Gaussian Process Regression (GPR) for modeling compressive strength of high-performance concrete (HPC). This machine learning approach is utilized to establish the nonlinear functional mapping between the compressive strength and HPC ingredients. To train and verify the aforementioned prediction model, a data set containing 239 HPC experimental tests, recorded from an overpass construction project in Danang City (Vietnam), has been collected for this study. Based on experimental outcomes, prediction results of the GPR model are superior to those of the Least Squares Support Vector Machine and the Artificial Neural Network. Furthermore, GPR model is strongly recommended for estimating HPC strength because this method demonstrates good learning performance and can inherently express prediction outputs coupled with prediction intervals.
1. Introduction
In construction industry, high-performance concrete (HPC) has been widely used in high-rise building/infrastructure projects for its superior strength, durability, and workability which exceed those of normal concrete [1, 2]. Generally, special ingredients are employed to make these specially designed concretes to satisfy a combination of performance requirements. Furthermore, the compressive strength is often considered as the most important property of HPC; other concrete properties such as elastic modulus, water tightness, and impermeability appear to have direct relationships with compressive strength [3]. Hence, the compressive strength is commonly utilized as the main criterion in defining the required quality of concrete [4].
The compressive strength is determined through a standard uniaxial compression test. If the test result does not meet the designed strength, remediation actions must be undertaken. Furthermore, corrective actions for underground concrete structures, such as concrete piles or foundations, can be very costly. As a result, an accurate estimation of the compressive strength before the placement is a practical need of construction engineers.
As the relationships between concrete components and compressive strength are complex and highly nonlinear, mathematical modeling of HPC is very challenging and oftentimes inaccurate [5]. Consequently, traditional statistical methods are inadequate for modeling of HPC compressive strength. Herein, the main goal is to construct a system that can learn from a data set of different HPC mixes and can predict accurately the compressive strength based on the pattern of concrete components.
Accordingly, this research extends the body of knowledge by evaluating the capability of the Gaussian Process Regression (GPR) [6] for modeling compressive strength of HPC. GPR is an efficient and reliable learning approach for modeling nonlinear and complex functional mappings [7, 8]; therefore, an assessment of this model performance on HPC strength prediction is particularly useful for practicing engineers. Additionally, this study can also be viewed as a comparative work since the performance of the GPR is benchmarked against those of other powerful nonlinear modeling methods including the Artificial Neural Network (ANN) and the Least Squares Support Vector Machine (LSSVM) [9]. Moreover, a data set of 239 HPC experimental tests recorded during the construction phase of the Nga Ba Hue highway overpass project in Danang City (Vietnam) has been collected for this research. The subsequent parts of the paper are organized as follows: Section 2 reviews pertinent research works in the literature. In Section 3, the research method is presented, followed by the experimental results. The conclusion of this study is stated in the final section.
2. Related Works
Due to the importance of the research topic, HPC compressive strength modeling has been a very active research area and various artificial intelligence (AI) techniques have been applied to tackle the problem of interest. Based on previous studies, AI techniques have proved its superior capability over traditional modeling methods. ANN is the most common modeling method [10, 11]. Yeh and Lien [12] applied the Genetic Operation Trees to establish empirical formulas which are more accurate than the nonlinear regression but less accurate than ANN models. Chou et al. [13] compare different data-mining techniques to identify the fittest method.
Sophisticated AI based systems have also been developed to fit particular HPC data sets. Słoński [14] combined the ANN and the Bayesian evidence framework which helps to construct the ANN structure. Genetic Weighted Pyramid Operation Tree was developed by combining four individual Genetic Operation Trees with adaptive weights [15]; the method appears to be better than the individual tree. The K-nearest neighbor based regression was integrated with Differential Evolution by Ahmadi-Nedushan [3]; this work demonstrated that despite being simple, in certain learning circumstances, the instance-based regression method can outperform the ANN.
Cheng et al. [16] proposed a fuzzy Support Vector Machine (SVM) method which can be superior over the original SVM and be comparable to the ANN. Pham et al. [5] employ LSSVM and Firefly Algorithm to construct a hybrid model for HPC strength estimation. The capability of Genetic Programming based prediction models, which fuses the genetic algorithm and the symbolic programming, was investigated by Chen and Wang [17], Mousavi et al. [18], and Castelli et al. [19]. Extensive researches on ensemble learning of Erdal et al. [20] and Chou et al. [21] found that this form of learning may boost the predictive capability of individual AI models. As can be seen in the literature, the capability of GPR has rarely been investigated for the task of HPC strength modeling. Therefore, our study is an attempt to fill this gap in the literature.
3. Research Method
3.1. Data Set of HPC Experiments
This research employs a data set consisting of 239 testing results of HPC concrete specimens. All the experimental tests were performed with 15 cm cylindrical specimens of HPC prepared according to the Vietnamese standard (TCVN 3105: 1993), which is relatively similar to the American standard ASTM C39. The amounts of cement (Kg/m3), sand (Kg/m3), small coarse aggregate (Kg/m3), medium coarse aggregate (Kg/m3), water (liter/m3), and superplasticizer (liter/m3) are batch components employed for expressing properties of a concrete sample. It is noted that the concrete age of each sample is measured in day.
Statistical descriptions of HPC test are reported in Table 1. It is worth noticing that the small coarse aggregate has diameter ranging from 5 to 10 mm; the medium coarse aggregate ranges in diameter from 10 and 20 mm. In addition, the ratio of water-to-cement of the concrete mixes in the data set ranges from 0.27 to 0.46. Such low water-to-cement ratio necessitates the utilization of superplasticizer to enhance the concrete workability. The minimum and maximum compressive strength of concrete specimens are 23.6 and 85.2 MPa, respectively.
3.2. Gaussian Process Regression (GPR)
GPR presents a probabilistic, nonparametric supervised learning approach for generalizing nonlinear and complex function mapping hidden in data sets. This approach has recently received huge attention of researchers in various study disciplines [4, 22]. GPR is very efficient to handle nonlinear data due to the use of kernel functions. Furthermore, a significant advantage of GPR is that the model can provide a reliability response to an input data [23].
Given a training set , the input data is called the design matrix and is the vector of desired output. The main assumption of GPR is that the output is computed as [6, 24]where represents a homoscedastic noise for all sample .
In GPR methodology, the observations in the data set of interest are considered as a single point sampled from a multivariate Gaussian distribution. Moreover, it can be assumed that this Gaussian distribution has the mean of zeros. The covariance function dictates the relation of one observation to another observation. The squared exponential covariance function is often selected in GPR for the task of function approximation [22, 24]: where the maximum allowable covariance is defined as . It is noted that reaches this maximum allowable covariance only when is very closed to and therefore is almost perfectly correlated with . Meanwhile, denotes the length parameter of the kernel function. In addition, represents a Kronecker delta function; if and if .
Given the training data set, the ultimate goal of the learning process is to predict the output value of a new queried input pattern. To achieve such goal, it is necessary to establish three covariance matrices as follows:
Due to the assumption that the data is sampled from a multivariate Gaussian distribution, we have the following expression:
Since is proved to be generated from a multivariate Gaussian distribution with the mean of and the variance of , the estimated mean and variance of the predicted output are given as follows:
When the hyperparameters of the kernel function are specified, the model parameters, including and , can be determined by Bayesian inference. This task boils down to maximizing a log-posterior function [24]. After the training phase terminates, the GPR model can be employed for predicting unknown input data sample.
4. Experimental Result and Comparison
4.1. Estimating HPC Compressive Strength with Gaussian Process Regression
In this experiment, the data set of HPC testing samples has been divided into two sets: the training set (90%) used for model construction and the testing set (10%) employed for model testing. Prior to the training process, it is necessary to specify the hyperparameters of GPR model. These hyperparameters include the initial value for the standard deviation of the noise , the maximum allowable covariance , and the length parameter of the kernel function. Based on recommendations in Mathworks [25], the initial value for the standard deviation of the noise is set as follows:where denotes the standard deviation of the desired output in the training data set.
To select the maximum allowable covariance and the length parameter of the kernel function, we have carried out a model selection process. These two parameters are allowed to be varied within the following parameter set (ParSet):
The training set is further separated into two subsets: subset 1 (90%) and subset 2 (10%); and a grid search procedure described in Algorithm 1 is performed to identify the most suitable set of and . Subset 1 herein plays the role of a training set. The set of parameters accompanied with the most desirable prediction performance of the model when it is used to predict data in subset 2 is selected. To quantify the model prediction performance, the Root Mean Squared Error (RMSE) has been used. Based on the experiment, the values of and are selected to be 0.1 and 0.05.
| ||||||||||||||||||||||||||||
With the three aforementioned hyperparameters, the training process of GPR can be executed. Accordingly the constructed model is utilized to predict the data instances in the testing set. The prediction outcome of the GPR testing phase is illustrated in Figure 1. To express the model accuracy, besides RMSE, the Mean Absolute Percentage Error (MAPE) and the Coefficient of Determination () have also been employed. The experimental result is reported as follows: RMSE = 4.92; MAPE = 7.29%; and = 0.90. These outcomes demonstrate that GPR has successfully captured the nonlinear function that determines the mapping between input factors of ingredients and the output of HPC compressive strength. As shown in Figure 1, the GPR model’s outcomes have obtained a good fit to a straight line. Since GPR can express uncertainty associated with each predicted output, Figure 2 illustrates the GPR results of prediction interval with 95% level of confidence. It is noted that all actual compressive strengths are within the range of the lower and upper boundaries.
4.2. Result Comparison
In this section of the article, to better evaluate the performance of the GPR model, the ANN [26] and LSSVM [9, 27] are employed as benchmark methods. The reasons for selecting these two benchmark models are that ANN is widely accepted as an effective tool for nonlinear function approximation and this algorithm has been successfully employed for predicting concrete strength [10, 13]; LSSVM is also an advanced machine learning method featured by high modeling accuracy [28–31] and it has been recently used for modeling concrete compressive strength [5].
To establish an ANN model, number of neurons in the hidden layer should be determined in advance and this parameter significantly influences the ANN prediction capability. In order to specify an appropriate model structure for ANN, the hidden layer starts with seven neurons (which is equal to the number of input factors) and then gradually increased to the maximum value of 30 neurons. The log-sigmoid function is commonly employed as the activation function and the Levenberg-Marquardt algorithm is utilized to train the ANN [26, 32, 33]. Herein, the training set is also separated into two subsets: subset 1 (90% or equivalently 193 data samples) and subset 2 (10% or equivalently 22 data samples); the number of neurons in the hidden layer results in the best prediction outcome in the testing phase of ANN being chosen. On the other hand, the hyperparameters of LSSVM (the regularization parameter and the kernel function parameter) are automatically tuned by the Firefly Algorithm as described in [5].
As mentioned earlier, the data set is randomly divided into 2 sets: training set (90%) and testing set (10%). Accordingly, the training and testing sets consist of 215 and 24 cases, respectively. Nevertheless, to avoid the randomness in testing sample selection and to compare the performances of models reliably, a 10-fold cross validation process is performed [34, 35]. Accordingly, the whole data set is randomly divided into ten data folds in which each fold in turn serves as a testing set; and the performance of the three models (GPR, ANN, and LSSVM) can be quantified by averaging results of the ten folds. Because all of the subsamples are mutually exclusive, this cross validation process can reliably assess the GPR model and the other two benchmarking methods.
Prediction results of the GPR model and the two benchmark models obtained from the ten-fold cross validation process are reported in Table 2. It can be observed that the GPR has achieved the best prediction result in all of the performance evaluation criteria, followed by LSSVM and ANN. Particularly, in terms of RMSE, GPR achieves a 12.74% improvement compared with LSSVM and a 22.46% improvement compared with ANN.
The outcomes of the GPR, LSSVM, and ANN models attained from the cross validation process are graphically reported in Figures 3, 4, and 5, respectively. In these figures, the horizontal axis measures the actual compressive strength noted as ; meanwhile, the vertical axis measures the predicted compressive strength () obtained from the prediction phases of each model. The performances of models in these figures can be appraised graphically with the line of best fit and quantitatively with values; it is noted that a data point locating closely to the line of best fit indicates an accurate prediction outcome. Based on experimental results that are visually displayed in Figures 3, 4, and 5, it can be confirmed that the GPR model is best suited for modeling the data set at hand.
5. Conclusion
This research has investigated the capability of the GPR model for the task of HPC compressive strength prediction. To construct and verify the machine learning model, a data set of actual HPC compressive tests has been collected for this study. Based on experimental results, the GPR model has achieved the most desirable performance with comparatively low prediction errors (RMSE = 4.04, MAPE = 5.15%) and a high coefficient of determination = 0.90. These are very desirable because the modeling HPC strength is widely known to be a highly complex task.
One significant advantage of GPR over other benchmark methods is that the GPR can deliver estimated compressive strength coupled with prediction interval. This property is also of great usefulness for construction engineers to reliably assess the strength of HPC concrete mixtures. Therefore, the GPR model is recommended as a promising alternative to assist construction engineers in concrete mixture design.
Despite the aforementioned advantages of GPR, one limitation of the study is that the employed approach is a black-box prediction model; hence, this may impose certain hindrance for civil engineers to understand the model structure. In addition, the size of the current data set should be expanded by collecting more testing results of HPC samples to further enhance the generalization of the prediction model.
Therefore, future extensions of this research may include applications of GPR for solving other prediction/modeling tasks in civil engineering, investigation on the effects of novel covariance functions on the GPR model performance, and discovering new techniques to improve the model learning capability. On the other hand, studying the potentiality of other machine learning techniques with transparent model structures such as instance-based learning or regression trees to meliorate the model interpretation is also a worth-investigating research direction.
Competing Interests
The authors (Nhat-Duc Hoang, Anh-Duc Pham, Quoc-Lam Nguyen, and Quang-Nhat Pham) declare that there is no conflict of interests regarding the publication of this article.