Abstract

Using the method of Stochastic Gradient Boosting, ten SMO-SVR are constructed into a strong prediction model (SGBS model) that is efficient in predicting the breakdown field strength. Adopting the method of in situ polymerization, thirty-two samples of nanocomposite films with different percentage compositions, components, and thicknesses are prepared. Then, the breakdown field strength is tested by using voltage test equipment. From the test results, the correlation coefficient (CC), the mean absolute error (MAE), the root mean squared error (RMSE), the relative absolute error (RAE), and the root relative squared error (RRSE) are 0.9664, 14.2598, 19.684, 22.26%, and 25.01% with SGBS model. The result indicates that the predicted values fit well with the measured ones. Comparisons between models such as linear regression, BP, GRNN, SVR, and SMO-SVR have also been made under the same conditions. They show that CC of the SGBS model is higher than those of other models. Nevertheless, the MAE, RMSE, RAE, and RRSE of the SGBS model are lower than those of other models. This demonstrates that the SGBS model is better than other models in predicting the breakdown field strength of polyimide nanocomposite films.

1. Introduction

As one of material products that have been developed for a long time, polyimide film (PI film) has been mainly applied in high and new technology industries such as aerospace, machinery, electrical and electronics engineering, optical communication, LCD, automobile, precision instrument, gas separation, and microelectronics [1]. With the development of nanotechnology, nanoparticles of different sizes, percentage compositions, and components have been mixed with PI by more and more researchers to produce high-quality polyimide nanocomposite films [2, 3].

Breakdown field strength is an important characteristic parameter to characterize polyimide nanocomposite films. It can be calculated by (breakdown voltage)/(film thickness). Many researchers have already made studies and analysis on the breakdown field strength of nanocomposite films [46]. There are many factors that can impact the breakdown field strength, including the type of nanoparticle, dielectric constant, electric conductivity, coefficient of thermal conductivity, composition, nanoparticles’ size and specific area, and composite film thickness. In order to establish the knowledge base for the material property regarding breakdown field strength of polyimide nanocomposite films, large quantities of experiments have to be prepared and measure related characteristics. Nevertheless, it is also well known that getting properties data is very costly in terms of time and materials. It is for this reason that developing a fast and efficiency method to predict the breakdown field strength of polyimide nanocomposite films is very much in demand.

Intelligent computing and neuronal network have been widely applied in performance prediction, identification, and optimization of nanocomposite films. Yang et al. used a generalized regression neural network (GRNN) to predict the friction coefficient of film [7]. Cho et al. optimized the characteristics of ITO/Al/ITO multilayer films by advantages of neural network and genetic algorithm [8]. Bahramian made use of an artificial neural network to predict the growth rate of TiO2 nanostructured film [9]. Ensemble learning has now become a new hotspot of intelligent computing [1013]. By using ensemble, several weak classifiers can be constructed into a strong classifier. Some common methods of ensemble learning include boosting, bagging, and stacking. Boosting [14] keeps upgrading weights during data extraction and revises the weights of data set that have been classified wrong. In the end, several weak classifiers are obtained and can be constructed into a strong classifier. Bagging [15] is used the reiterative training to get several classifiers based on a training set. Stacking [16] contains two layers. In the first layer, different algorithms are employed to generate several weak classifiers. At the same time, a new data set with the same size of the original data set is also generated. Then, the new data set together with a new algorithm can be used to construct the classifier of the second layer.

The purpose of this paper is to develop a Stochastic Gradient Boosting + SMO-SVR model (SGBS model) to predict the breakdown field strength of polyimide nanocomposite films. In what follows, film preparation and prediction model are introduced first. The experimental details for sputtering systems and materials are described next. Then, the experimental results are described and the establishment of SGBS model with 10-fold cross validation results is carried out. Comparison experiments between linear regression, BP neural network (BP), general regression neural network (GRNN), SVR (support vector regression), and SMO-SVR models are conducted.

2. Nanocomposite Film Preparation and Prediction Model

2.1. Preparation of Nanocomposite Films

The method of in situ polymerization is used to prepare the polyimide matrix inorganic nanocomposite film. Experimental materials include 4,4′-diaminodiphenyl ether (ODA), pyromellitic dianhydride (PMDA), dimethylacetamide (DMAc), Al2O3, rutile TiO2, BaTiO3, SiO2, and ethanol. The detailed purchase information of the above experimental materials is shown in Table 1. Firstly, put PMDA into the solution of ODA in DMAc to produce an amount of polyamide acid of certain viscosity. Secondly, add in different nanoparticles. Finally, let the mixture go through paving membrane heat treatment and imidization transform.

To verify our SGBS model, we have designed and prepared in this paper nanoparticle samples with different types, sizes, ratios, and thicknesses. The SEM image of surface appearance of pure PI is given in Figure 1(a). The surface of film is smooth and its tightness is well. Figure 1(b) shows the SEM image of surface appearance of PI/BaTiO3 composite film doped with content of 60 wt%. There are large amounts of BaTiO3 nanoparticles exposing on the surface of film with the size from 100 nm to 300 nm. Comparing with the other films, the surface appearance of film has been changed to be rough due to the dopant of BaTiO3 nanoparticles. Its density has also been decreased. Figure 1(c) shows the SEM image of surface appearance of PI/TiO2 composite film doped with content of 5 wt%. TiO2 can absorb ultraviolet ray to change the color of film into brown. The surface appearance of film is smooth compared with pure PI film. Figure 1(d) shows the SEM image of surface appearance of PI/Al2O3 composite film doped with content of 20 wt%. There are some nanoparticles appearing in the surface of film, uniformly, where the diameter of nanoparticles is observed to be 70 nm and some particles are not cluster. In conclusion, the prepared hybrid PI particles are evenly distributed.

2.2. Standard SMO-SVR Model

SVM (support vector machine) [17] is one of the most typical machine learning methods in field of statistical learning theory. It has a very good learning ability in a small sampling space. Compared with BP, GRNN, and other neural networks, it has advantages such as structural risk minimization, global optimization solution, high dimensional space, and being linearly separable. It has been widely used for recognitions and regressions.

The core of support vector regression (SVR) is to find out a hyperplane, namely, to a minimized linear function with inequality constraints, giving a training sample where means input and means output. The training sample needs to be turned into a linearly separable problem when it is a linear inseparability. Generally, the sample space can be mapped from a low-dimensional space into a high dimensional space by using a kernel function. The nonlinear regression function (estimation) is where is a threshold, is a weight, and is the nonlinear mapping function. The loss function is shown as follows:By importing a Lagrange multiplier, the minimization of the objective function can be expressed asTransform (4) into a dual problem, and we get where is a penalty factor and and are Lagrangians; then the regression function is where is the number of support vectors in (6) [18]. In this paper, a normalized polynomial kernel has been taken as the kernel function.

John Platt, from Microsoft Research, proposed the SMO (sequential minimal optimization) algorithm in order to shorten the training time of support vector regression in 1998 [19]. SMO can optimize the value of two samples at one time. Through the loop iteration with a given times , the value of all samples can be optimized, as shown in Figure 2. For the data in Table 2, the method of 10-fold cross validation is used to make the model training on the standard SVR and SMO-SVR. The time of modeling SVR and SMO-SVR is 0.26 s and 0.03 s, respectively. SMO can promote the training efficiency of support vector regression while reducing the training time of the model.

2.3. Promoting the Model by Stochastic Gradient Boosting

Boosting [20], as one of the most important ensemble learning methods, is to obtain a predictive function by construction and ensemble of a series of predictive functions. The core idea of gradient boosting, proposed by Friedman [21], is to construct an ensemble learning machine by calculating a loss function and letting the function descend along its gradient. In other words, it is to calculate the loss function of previous model, so as to build a new model along the descending gradient direction of the loss function. Eventually, a regression model can be generated through times of iterations. In this paper, we use the Stochastic Gradient Boosting to turn ten SMO-SVR models into a strong predictive model (SGBS model). The algorithmic steps of the SGBS model are shown as follows:(1) is a data sample of breakdown field strengths of different nanocomposite thin films, in which is input and is output. The loss function is , and the prediction model of SMO-SVR is .(2)Initialize the model(3)Calculate the upper limit of iterations by cross validation.(4)Calculate the gradient direction(5)Fit the SMO-SVR by using least squares to obtain the fitting model .(6)Figure out new step length based on the loss function(7)Generate a new model(8)Stochastically extract of the training sample to fit the SMO-SVR during every time of iteration; then we can get the SGBS model after times of iterations:

3. Experiment and Result Analysis

3.1. Construction of the Experimental Sample

Test the breakdown field strength by using a withstanding voltage tester (type: CS2674C). Voltage range is 0–50 kV; test error is ±5%; leakage current measurement range is 0.5–20 mA; measurement error is ±5%. Take polymethylphenylsiloxane fluid with high insulation strength (which can be up to 16 kV/mm) as the medium, with the rate of voltage rise of 500 V/s. Test the breakdown field strength of the nanocomposite films prepared in Section 2.1. Select 20 samples as a group for testing each hybrid PI, and take the mean value of the 10 middle breakdown field strengths as the breakdown field strength of the group. Then, calculate the standard deviation. After the experiment we get the data of breakdown field strengths as shown in Table 2. The input includes type , doping ratio , electrical resistivity , dielectric constant , thermal conductivity , size , and specific area of nanoparticles and thickness of the film . The output is the breakdown field strength of the hybrid PI.

3.2. Evaluation Indicators of the Prediction Performance

There are many indicators for evaluating the prediction performance, such as MAPE (mean absolute percentage error), MAE (mean absolute error), RMSE (root mean squared error), CC (correlation coefficient), RAE (relative absolute error), and RRSE (root relative squared error). In this paper, CC, MSE, RMSE, RSE, and RRSE are taken as evaluation indicators.

CC (correlation coefficient) represents the relevancy of the linear regression relationship between (independent variable) and (response variable) of the model. It is a coefficient of a simple linear correlation between and its estimated values. CC is larger than zero and ranges from 0 to 1. A larger CC means a more relevant linear regression relationship.

MSE (mean absolute error) is to judge the different degree between predicted values and real ones. It is inversely proportional to prediction accuracy. A smaller MSE means a better effect the predictor can be with. It can be expressed as where represents predicted values and represents real values.

RMSE (root mean squared error) is the square root of the ratio of the quadratic sum of deviations between predicted values and real values to the times of predictions. It is sensitive to maximum or minimal errors of a group of predicted values and therefore can well reflect prediction accuracy. RMSE is inversely proportional to the prediction accuracy. The smaller the RMSE is, the more accurate the predictor can be. It can be expressed as follows:

RSE (relative absolute error) is the different degree between the absolute deviation obtained from the prediction model and the absolute deviation obtained by directly speculating the training sample. It is inversely proportional to prediction accuracy. The smaller the RSE is, the higher the prediction accuracy can be:

RRSE (root relative squared error) can be calculated as follows:

RRSE is also inversely proportional to prediction accuracy. The smaller the RRSE is, the higher the prediction accuracy can be [22].

3.3. Experimental Results and Analysis

To verify the SGBS prediction model, the type, dielectric constant, electrical resistivity, thermal conductivity, size and specific area of nanoparticles, and the thickness of films in Table 2 are taken as the input , and the breakdown field strength of hybrid PI is taken as the output . In this paper, we use Macbook Pro (CPU: Intel I7-2640 M; memory: 16G) as the hardware for experiments and use Matlab 2012a to program prediction model.

Experiment 1. Use the method of 10-fold cross validation to fit the data in Table 2. Main parameters of the SGBS model include kernel function, and , of which is the penalty factor of SMO-SVR and is the extracting ratio when training the sample. First of all, normalize the sample data. Three kernel functions—normalized polynomial kernel, polynomial kernel, and RBF kernel—are employed to test the sample. The results are listed in Table 3. From the table we know that the normalized polynomial kernel has the highest correlation coefficient but lowest mean absolute error, root mean squared error, relative absolute error, and root relative squared error. Therefore, we choose it as the kernel function for the SGBS model.
For optimization function, the penalty factor of outliers is selected by experience as well as by experiments. Figure 3 shows the comparison on prediction performances of SGBS model when varies from 1 to 1.8. Figure 3(a) demonstrates that the correlation coefficient reaches its peak values when , 1.7, and 1.8, which are chosen as the value of , for correlation coefficient is proportional to prediction accuracy. By analyzing Figures 3(b) and 3(d) we know that the mean absolute error and relative absolute error reach their minimal values when , only larger than the values when and 1.8. Therefore, they reach optimal performance when , for mean absolute error and relative absolute error are inversely proportional to prediction accuracy.
By analyzing Figures 3(c) and 3(d) we know that the smallest root mean squared error and root relative squared error appear at , followed by and . Based on the above analysis, this paper takes as the optimum value.
is the specify shrinkage rate in Stochastic Gradient Boosting, namely, the proportion of the stochastically extracted training sample. Figure 4 shows the impact of variation of on prediction performance when . When , correlation coefficient reaches its peak value, and the prediction attains the best performance as shown in Figure 4(a). By analyzing Figures 4(b)4(e) we know that when , mean absolute error, root mean squared error, relative absolute error, and root relative squared error reach their minimal values, signifying the best prediction performance.
According to the above analysis we know that when and are taken as the optimum values of the SGBS model, the prediction performance is the best. Figure 5 shows the degree of fitting between predicted values and real ones. Figure 6 shows the absolute error ratio of prediction. The tables and figures demonstrate that the error ratios of sample 8 (70 wt%, 100 nm, BaTiO3), sample 17 (15 wt%, 40 nm, SiO2), and sample 30 (15 wt%, 7 nm, SiO2) are all larger than 15%, indicating an ordinary fitting, while the error ratios of other samples are less than 15%, indicating a better fitting. For all the six multicomponent TiO2 film samples with the thickness of 35 μm, the prediction errors are not larger than 15%. For the eight BaTiO3 film samples, there is one sample of which the prediction errors are larger than 15%. For all the nine multicomponent Al2O3 film samples, the prediction errors are not larger than 15%, while for the nine multicomponent SiO2 film samples, there are also two samples of which the prediction errors are larger than 15 wt%. In this model, the prediction performance of the multicomponent Al2O3 and rutile TiO2 with different thicknesses is better than that of the BaTiO3 and SiO2 film samples. For the thirty-two nanocomposite films with different components, mixtures, and thicknesses, there are twenty-nine film samples of which the prediction errors are lower than 15%, proving that the model is of practical value in actual engineering works.

Experiment 2. In order to further verify the SGBS model, it needs to be compared to other models, namely, the linear regression, BP neural network, GRNN neural network, SVR (support vector regression), and SMO-SVR, under the same conditions. Comparison results of the prediction performance of these models are shown in Table 4. The correlation coefficient of SGBS model is 0.962, larger than that of the models of linear regression [23], BP neural network [24], GRNN neural network [7], SMO-SVR [25], and SVR [26], proving that the linear regression relationship of the SGBS model is better than that of the other five models. In the SGBS model, the root mean squared error and root mean squared error are 20.8668 and 26.520%, respectively, lower than those of the other models, certifying its better prediction performance.

Experiment 3. In order to validate the generosity of the model, the actual measurement results conducted by Shi et al. in [27] are chosen to do the prediction. In this preference, the particle size is 30 μm, and the doping ratio is 2 wt% with PI/nano-Al2O3 composite films. The thickness of film is 30 μm. Its breakdown field strength is 233 kV/mm. Adopting the samples in Table 2 as the training set of SGBS model. For the parameters of the model, , . Using the 2 wt%-PI/nano-Al2O3 composite films in [27] as testing sample, the results could be obtained as shown in Table 5. For the test result of breakdown field strength that is 233 kV/mm, its prediction value is 225.9667 kV/mm by SGBS model. The error ratio is 3.0185%. The MAE, RMSE, RAE, and RRSE are 7.0334, 7.0334, 24.5976, and 24.5976. The prediction and actual data are in good agreement. This method could predict the PI/nano-Al2O3 composite films effectively.

4. Conclusions

This paper presents an ensemble learning method for predicting breakdown field strength of polyimide nanocomposite films. By using the method of Stochastic Gradient Boosting, ten SMO-SVR prediction models are constructed into a strong prediction model (SGBS model) that is efficient in predicting the breakdown field strength. Through analyzing the experiment data we obtain following conclusions:(1)In prediction of thirty-two nanocomposite films of different components, particles, and thicknesses (25–30 μm) by using the method of 10-fold cross validation, there are twenty-nine samples of which the prediction errors are lower than 15%, proving that the SGBS model is efficient in predicting the breakdown field strength of polyimide nanocomposite films.(2)Comparisons show that the SGBS model has a larger correlation coefficient than that of linear regression, BP, GRNN, SVR, and SMO-SVR models but smaller root mean squared error and root relative squared error. Hence, prediction performance of the SGBS model is better than that of the other five models.(3)The SGBS model shows a better prediction on Al2O3 and rutile TiO2 films than on BaTiO3 and SiO2 films.

Next, some other ensemble learning methods will be employed to predict the corona resistance, dielectric constant, dielectric constant, and thermal properties of polyimide nanocomposite films.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This paper has obtained the support of the National Natural Science Foundation of China (51077028, 51307046, and 60803096) and Heilongjiang Natural Science Foundation of China (A201006), National Basic Research Program of China (2012CB723308), Fund of the State Ethnic Affairs Commission of China, and the Fundamental Research Funds for the Central Universities. Authors also gratefully acknowledge the helpful comments and suggestions of the reviewers who improved the presentation.