Predicting Ink Transfer Rate of 3D Additive Printing Using EGBO Optimized Least Squares Support Vector Machine Model
Ink transfer rate (ITR) is a reference index to measure the quality of 3D additive printing. In this study, an ink transfer rate prediction model is proposed by applying the least squares support vector machine (LSSVM). In addition, enhanced garden balsam optimization (EGBO) is used for selection and optimization of hyperparameters that are embedded in the LSSVM model. 102 sets of experimental sample data have been collected from the production line to train and test the hybrid prediction model. Experimental results show that the coefficient of determination (R2) for the introduced model is equal to 0.8476, the root-mean-square error (RMSE) is 6.6 × 10 (−3), and the mean absolute percentage error (MAPE) is 1.6502 × 10 (−3) for the ink transfer rate of 3D additive printing.
The quality of the vamp printed by the traditional running table depends entirely on the using experiences of the printer, so the printing quality stability is poor when different people use the printer. A good way to get better quality and stability of printing is intelligent parameter adjusting of the 3D additive printing machine. In the practical production, squeegee angle, printing pressure, squeegee speed, ink viscosity, and screen mesh thickness affect the adhesion of ink from screen to sneaker surface and then directly affect the printing quality of vamp . Adjusting all of these influencing factors that are parameters of the printing machine should be important and can be considered in the production process of 3D additive printing. Therefore, the study on the optimization of parameter adjusting of 3D printing machine is of great significance to realize the high-precision printing of 3D printing machine and improve quality and stability of printing.
In the case of unchanged printing screen, there are four adjustable parameters of the printing machine, such as the squeegee angle, printing pressure, squeegee speed, and ink viscosity . In the practical production, the additive printing process (that is, the ink adhesion through the screen surface attached to the process) directly determines the printing quality of the product. Generally the product quality is higher when ink adhesion is more; the product quality is lower when ink adhesion is less. Therefore, ink transfer rate (ITR) can be used as an evaluation standard to measure the quality of printing. Therefore, it is necessary to establish a prediction model based on experimental data to obtain the optimal process parameters .
Based on the above analysis, the process parameter optimization problem of 3D additive printing can be described as the mathematical model as following. The squeegee angle, printing pressure, squeegee speed, and ink viscosity are four input parameters in the model, ITR is the output of the printing effect index, and the goal of the model is finding out the relationship between the input parameters and output index . According to this model, the optimal parameter combination can be found from input parameter combination evaluating. In 3D additive printing, the model which builds the relationship between input parameters and output index has become the important and difficult optimization problems. As shown in previous studies , the difficulty is that the mapping between the input parameters and the printing effect index is highly nonlinear.
Because of the importance of the research topic, the production staff tried to do a lot of work on the experiment of improving the printing quality by the combination of process parameters. Machine learning as an advanced modeling tool has been used effectively in engineering applications [6–12]. Wang et al.  divided the four process parameters into five levels to design the orthogonal experiment and divided the printing quality evaluation into five levels to obtain a group of relative optimal process parameter combination through comparative analysis. On the basis of the former research, Wang et al.  used the artificial neural network (ANN) model to study the experimental database of 102 production test results and built a network model for quality prediction.
According to the literature review, the application of advanced machine learning-based model to estimate printing quality remains a rarity. Although other methods such as ANN and gene expression programming (GEP) can also be used for printing quality modeling, these methods also have certain difficulties . The neural network model is established by using the gradient descent method and the back propagation algorithm, which means that their training stage is easy to fall into the local optimum. In addition, although GEP can automatically construct prediction equations, the prediction accuracy of this machine learning method may not be as good as that of ANN. Therefore, the paper is to study the ability of other advanced machine learning methods, which should be studied to improve prediction accuracy.
In this study, the least squares support vector machine (LSSVM) is used to construct a functional mapping to solve the above difficult printing quality prediction problem . As a powerful nonlinear and multivariate modeling tool, LSSVM is used to solve engineering mathematical problems . However, determining an appropriate set of LSSVM model hyperparameters can be a challenging task because of the myriad of solution candidates . Since the setting of LSSVM model hyperparameters can be modeled as optimization tasks, the garden balsam optimization (GBO) algorithm is used. On this basis, the performance of the LSSVM and GBO hybrid model in printing quality modeling is studied. The reason why GBO is chosen in this study is that GBO is a relatively new algorithm with good optimization performance . 102 data samples have been collected from the experiment, including four input factors, such as printing pressure, squeegee angle, squeegee speed, and ink viscosity, to train and test the LSSVM model. In addition, since most of the previous work on printing quality estimation only relied on individual machine learning algorithms, one of the main contributions of this study is proposing a hybrid model to improve the accuracy by mixing machine learning and swarm intelligence optimization methods.
The chapters of this paper are organized as follows: the next part reviews the calculation methods used by LSSVM and GBO to build the hybrid prediction model. The third part describes the proposed model. The fourth part reports the experimental results. The concluding remarks are provided in the last section.
2. The Employed Computational Intelligence Methods
2.1. Least Squares Support Vector Machine (LSSVM)
This section describes the LSSVM method used to construct a mapping function between the ITR and technological parameters. The LSSVM is a powerful nonlinear function approximation method that can effectively process multivariable and small-scale datasets . This machine learning approach first performs data transformation, mapping data from the original input space to the higher-dimensional feature space . Therefore, a linear model can be constructed in the eigenspace to infer the mapping relationship between response variables and a set of independent variables.
In addition, the radial basis function (RBF) kernel is commonly used in LSSVM . It is worth noting that in addition to the RBF kernel, other functions such as linear or polynomial kernels can also be applied. However, in previous applications, the RBF kernel has been shown to have satisfactory learning performance . Therefore, this paper chooses this kernel function to study.
LSSVM is a mature technology, and I will not go into details here. The model for the LSSVM method requires setting two hyperparameters, such as the regularization coefficient (γ) and the kernel function parameter (σ). The randomness of these two parameters is relatively large, and there is no certain law to follow. Intelligent optimization algorithms can be used to solve this problem [21, 22].
2.2. Enhanced Garden Balsam Optimization (EGBO)
Garden balsam optimization (GBO) is a recent swarm-based evolutionary algorithm that is inspired by the seed transmission mode of garden balsam. Garden balsam randomly ejects the seeds within a certain range by virtue of mechanical force originating from cracking of mature seed pods, which is different from natural expansion of most species of plants. The seeds scattered to suitable growth area will have greater reproductive capacity in the next generation, followed by iteration until the most suitable point for growth in a particular space is eventually found. Like other evolutionary algorithms, GBO is a numerical random search algorithm that simulates natural behavior. However, it also shows some deficiencies in the experiment. In the iteration process of the basic GBO algorithm, there is no cooperative mechanism between individuals; furthermore, there is lack of utilization of optimal individual information.
In the algorithm improvement, the balance between the early global exploration capability and the later local development capability should be considered. Enhanced garden balsam optimization (EGBO) uses flower pollination strategy for the population [23, 24]. This flower pollination strategy depends on the strength of the pollination. There are two key steps in this strategy: global pollination and local pollination.
In the global pollination step, flower pollens are carried by pollinators such as insects, and pollens can travel over a long distance. This ensures the pollination and reproduction of the most fit, and thus, we represent the most fit as . The first rule can be formulated as follows:where is the solution vector at iteration t and is the current best solution. The parameter is the strength of the pollination which is the step size randomly drawn from Lèvy distribution . This paper draws from a Lèvy distribution as follows:where is the standard gamma function, and this distribution is valid for large steps. In all our simulations below, we have used .
The local pollination and flower constancy can be represented as follows:where and are pollens from the different flowers of the same plant species. This essentially simulates the stability of the flower in a finite neighborhood. Mathematically, if and come from the same species or selected from the same population, this becomes a local random walk if we draw from a uniform distribution in [0, 1]. To start with, we can use as an initial value and then do a parametric study to find the most appropriate parameter range. From our simulations, we found that works better for most applications.
The above two key steps plus the switch condition can be summarized in the pseudocode shown in Algorithm 1.
Based on the strategy, the EGBO algorithm flowchart is given in Figure 1. The detailed steps of the algorithm are as follows.
3. Framework of EGBO-LSSVM Model
This section describes the framework of the hybrid model used to predict the final ink transfer rate. The hybrid model combines EGBO and LSSVM. It is worth noting that LSSVM is used to build a functional map that calculates the final ink transfer rate value based on four input variables. Since the regularization coefficient and kernel function parameters need to be determined in the training phase of the LSSVM model, the EGBO clustering intelligent algorithm is adopted to set these two hyperparameters automatically.
Therefore, the hyperparameters of the LSSVM model are randomly generated within the above boundary, and their expressions are as follows:where is the i-th hyperparameter of the LSSVM model. represents uniformly random numbers generated between 0 and 1. = 0.01 and = 1000 are the lower and upper bounds of the hyperparameter, respectively.
Figure 2 shows the overall concept of the hybrid EGBO-LSSVM model.
In order to determine the most appropriate LSSVM’s hyperparameter set, k-fold cross validation is used in this study. To allow for the calculation of costs, let k = 5. Based on the cross-validation framework, the 102 sample data sets are divided into 5 data folds. The LSSVM predictive model is evaluated 5 times with each set of hyperparameters obtained by the EGBO algorithm. In each evaluation time, four data folds are used for model training, and the remaining one data fold is used for model prediction. The fitness function values are as follows:where Rmk denotes root-mean-square error (RMSE) of LSSVM.
RMSE is calculated as follows:where YA denotes actual value, YP denotes predicted value, and NK denotes the number of samples.
After calculating the cost function for each member of the population, the EGBO algorithm performs mechanical and secondary propagators to explore the search space and find a better solution, and then updates the positions of all population members based on the elitist-random selection operator. The EGBO optimization continues until the number of iterations reaches the value of itermax. The optimized LSSVM model can be used to predict the ink transfer rate of new data samples when the appropriate set of hyperparameters is determined.
4. Experimental Results and Comparison
This section will present and analyze the experimental results of the hybrid EGBO-LSSVM model in the ink transfer rate prediction experiment. Four variables, including printing pressure (X1), squeegee angle (X2), squeegee speed (X3), and ink viscosity (X4), have been selected as input factors, and ink transfer rate (Y) has been selected to represent the printing quality of 3D additive printing machine. This data set has been summarized and recorded by Wang et al. . Based on the literature review and available data, the assumptions of this study are as follows: (1) the ink transfer rate can be adequately modeled using the above four variables. (2) The current number of data samples is sufficient to meet the model construction and verification process.
The three variables of printing pressure (X1), squeegee angle (X2), and squeegee speed (X3) can be set and directly obtained on the 3D additive printing machine. Ink viscosity (X4) is obtained by instrument detection before experiment. Ink transfer rate (Y) is obtained by the difference in pulp weight before and after printing.
Table 1 gives the statistical description of the four influencing factors and ultimate binding strength. The scatter plot of each input variable and ink transfer rate is shown in Figure 3. In addition, the entire data set is summarized in Table 2 of this paper.
The experimental work for all problems is done on DELL Inspiron computer with Intel(R) Core (TM) i7-4500U, 2.4 GHz processor and 8 GB of memory running Windows 10. The implementation of experiment is done in MATLAB R2014a tool. EGBO and LSSVM parameters are shown in Table 3.
The model optimization process by EGBO is shown in Figure 4. After 100 iterations, the optimal hyperparameter of the LSSVM prediction model is determined as follows: γ = 3.0373 and σ = 4.926.
After further analysis, the data set containing 102 samples is divided into training set (81 samples) and test set (21 samples). After the model is stabilized, the training set samples and test set samples are predicted to obtain the prediction results of ink transfer rate and the real value comparison. Figures 5–8 evaluate the mode’s performance on determining ink transfer rate of 3D additive printing based on optimal parameters. The comparison between estimated ink transfer rate and measured ink transfer rate in training and testing phases is shown in Figure 5. As can be seen from the figure, most of the predicted values are close to the actual values. The absolute error and absolute percentage error between estimated and measured ink transfer rate in testing phase are shown in Figure 6. As can be seen from the figure, the absolute error predicted by the model is within 0.01, and the absolute percentage error is within 0.03.
In order to further analyze the performance of the prediction model, another analysis method is used. Figure 7 reports the scatter plot of estimated and measured ink transfer rate in training and testing phases. As shown in Figure 7, we estimated ink transfer rate versus the measured data rise in a straight line with a slope of unity and there is a high number of the estimated data points in the vicinity of the line. In addition, the probability distribution of errors in the model training stage and the test stage is illustrated in Figure 8. As can be seen from the figure, the probability distribution of the error is basically a normal distribution with mean 0.
In order to better verify the capability of the EGBO-LSSVM model, the performance of the hybrid model is compared with that of BPANN, MARS, and RegTree [25–27, 28]. BPANN, MARS, and RegTree are effective machine learning methods for modeling nonlinear and multivariate data [29–32].
As mentioned earlier, the data set containing 102 samples is divided into a training set (81) and a test set (21).
In order to offset the randomness of data selection, the EGBO-LSSVM model is executed 20 times and the average results obtained are compared with the other models for the same number of runs. To evaluate the performance of the EGBO-LSSVM model, relative percentage error (MAPE) and determination coefficient (R2) are used in addition to the RMSE described above [33, 34].
The experimental results of ink transfer rate prediction of the four models are reported in Table 4. From this table, it can be see that EGBO-LSSVM is performing optimally (RMSE = 0.0066, MAPE = 1.6502%, and R2 = 0.8476), followed by MARS (RMSE = 0.0080, MAPE = 2.0296%, and R2 = 0.7941), RegTree (RMSE = 0.0084, MAPE = 2.0915%, and R2 = 0.7563), and BPNN (RMSE = 0.0097, MAPE = 2.3631%, and R2 = 0.7467). It is worth noting that these results are the average of 20 repeated data samples used for model predictions. In addition, the prediction error boxplot of all models is shown in Figure 9.
It is also important to find out statistical significance of EGBO-LSSVM over other comparative models. To quantify the performance of the models statistically, the Wilcoxon symbolic rank test is used to better confirm the statistical significance of the superiority of EGBO-LSSVM. The p value calculated by the test is shown in Table 5 (preselection threshold = 0.05). The results depict the pairwise value obtained from the Wilcoxon symbolic rank test for all models. It is observed from the results that EGBO-LSSVM and MARS are statistically similar for obtaining RMSE and MAPE values. In addition to that, a significant statistical difference has been observed in the results of the EGBO-LSSVM and other competitive models for obtaining the RMSE, MAPE, and the R2 values.
In addition, to assess the sensitivity of input variables to the performance of the EGBO-LSSVM model, the Fourier amplitude sensitivity test (FAST) [35, 36] is used in this study. This study relies on a toolkit developed by Pianosi et al. [37, 38] to implement the FAST method. Based on the fast method, the variation of ink transfer rate prediction is decomposed into partial variances of input factors by Fourier transform. The influence of input characteristics on the output of EGBO-LSSVM can be quantified by the first-order sensitivity index (FOSI) . The sensitivity analysis results are shown in Figure 10.
As can be seen from Figure 10, X1 (FOSI = 49.32%) has the greatest impact on the model prediction results, followed by X4 (FOSI = 20.32%), X2 (FOSI = 18.45%), and X3 (FOSI = 11.49%). From the analysis results, it can be seen that all the characteristics have certain influence on the prediction of EGBO-LSSVM.
In this study, a model is developed and proposed in order to predict ink transfer rate in 3D additive printing by printing pressure, squeegee angle, squeegee speed, and ink viscosity as influential parameters. The conclusion can be summarized as follows:(1)The high viability and capability of LSSVM approach with RBF kernel to calculate ink transfer rate in 3D additive printing are proven based on the available experimental data.(2)Hyperparameters, including γ and σ, have a significant effect on LSSVM training results and generalization ability. By applying the EGBO algorithm, the optimal values of γ and σ are found to be equal to 3.0373 and 4.926, respectively.(3)A hybrid model of LSSSVM and EGBO provides an appropriate tool to predict ink transfer rate in 3D additive printing. The outputs of the model had good agreement with the experimental data. The determination coefficients, the root-mean-squared errors, and the mean absolute percentage errors of the model are 0.8476, 6.6 × 10 (−3), and 1.6502 × 10 (−3), respectively.
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This work was partially supported by the Natural Science Foundation of China (5147509) and the National Key R&D Program of China (2018YFB1308800).
Y. Wang, Y. Liu, and Y. Sun, “A hybrid intelligence technique based on the Taguchi method for multi-objective process parameter optimization of the 3D additive screen printing of athletic shoes,” Textile Research Journal, vol. 90, no. 9-10, pp. 1067–1083, 2020.View at: Publisher Site | Google Scholar
D.-K. Bui, T. Nguyen, J.-S. Chou, H. Nguyen-Xuan, and T. D. Ngo, “A modified firefly algorithm-artificial neural network expert system for predicting compressive and tensile strength of high-performance concrete,” Construction and Building Materials, vol. 180, pp. 320–333, 2018.View at: Publisher Site | Google Scholar
S. Faizollahzadeh Ardabili, B. Najafi, S. Shamshirband, B. Minaei Bidgoli, R. C. Deo, and K.-w. Chau, “Computational intelligence approach for modeling hydrogen production: a review,” Engineering Applications of Computational Fluid Mechanics, vol. 12, no. 1, pp. 438–458, 2018.View at: Publisher Site | Google Scholar
Y. Wang, Y. Liu, Z. Meng, and Y. Sun, “Optimization of process parameters for 3D additive screen printing based on genetic algorithm and neural network,” Journal of Textile Research, vol. 40, no. 11, pp. 168–174, 2019.View at: Google Scholar
M. H. Ahmadi, M. A. Ahmadi, M. A. Nazari, O. Mahian, and R. Ghasempour, “A proposed model to predict thermal conductivity ratio of Al2O3/EG nanofluid by applying least squares support vector machine (LSSVM) and genetic algorithm as a connectionist approach,” Journal of Thermal Analysis and Calorimetry, vol. 135, no. 1, pp. 271–281, 2019.View at: Publisher Site | Google Scholar
S. Li and Y. Sun, “Garden balsam optimization algorithm,” Concurrency & Computation Practice & Experience, vol. 32, no. 2, 2020.View at: Google Scholar
J. Suykens, J. V. Gestel, J. D. Brabanter, B. D. Moor, and J. Vandewalle, Least Squares Support Vector Machines, World Scientific, Hackensack, NJ, USA, 2002.
M. A. Al-Betar, M. A. Awadallah, I. A. Doush, A. I. Hammouri, M. Mafarja, and Z. A. A. Alyasseri, “Island flower pollination algorithm for global optimization,” The Journal of Supercomputing, vol. 75, pp. 1–44, 2019.View at: Google Scholar
P. Taherei Ghazvinei, H. Hassanpour Darvishi, A. Mosavi et al., “Sugarcane growth prediction based on meteorological parameters using extreme learning machine and artificial neural network,” Engineering Applications of Computational Fluid Mechanics, vol. 12, no. 1, pp. 738–749, 2018.View at: Publisher Site | Google Scholar
A. Gholampour, I. Mansouri, O. Kisi, and T. Ozbakkaloglu, “Evaluation of mechanical properties of concretes containing coarse recycled concrete aggregates using multivariate adaptive regression splines (MARS), M5 model tree (M5Tree), and least squares support vector regression (LSSVR) models,” Neural Computing & Applications, vol. 32, pp. 295–308, 2018.View at: Publisher Site | Google Scholar
A. T. C. Goh, W. Zhang, Y. Zhang, Y. Xiao, and Y. Xiang, “Determination of earth pressure balance tunnel-related maximum surface settlement: a multivariate adaptive regression splines approach,” Bulletin of Engineering Geology and the Environment, vol. 77, no. 2, pp. 489–500, 2018.View at: Publisher Site | Google Scholar
V. H. Nhu, N. D. Hoang, V. B. Duong, H. D. Vu, and D. Tien Bui, “A hybrid computational intelligence approach for predicting soil shear strength for urban housing construction: a case study at Vinhomes Imperia project, Hai Phong City,” Engineering with Computers, vol. 36, no. 2, pp. 603–616, 2019.View at: Publisher Site | Google Scholar
D. Tien Bui, V.-H. Nhu, and N.-D. Hoang, “Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and multi-layer perceptron neural network,” Advanced Engineering Informatics, vol. 38, pp. 593–604, 2018.View at: Publisher Site | Google Scholar