Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2016, Article ID 5471748, 9 pages
http://dx.doi.org/10.1155/2016/5471748
Research Article

Research on a Novel Kernel Based Grey Prediction Model and Its Applications

School of Science, Southwest University of Science and Technology, Mianyang, China

Received 2 July 2016; Revised 23 October 2016; Accepted 14 November 2016

Academic Editor: Michele Betti

Copyright © 2016 Xin Ma. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The discrete grey prediction models have attracted considerable interest of research due to its effectiveness to improve the modelling accuracy of the traditional grey prediction models. The autoregressive GM model, abbreviated as ARGM, is a novel discrete grey model which is easy to use and accurate in prediction of approximate nonhomogeneous exponential time series. However, the ARGM is essentially a linear model; thus, its applicability is still limited. In this paper a novel kernel based ARGM model is proposed, abbreviated as KARGM. The KARGM has a nonlinear function which can be expressed by a kernel function using the kernel method, and its modelling procedures are presented in details. Two case studies of predicting the monthly gas well production are carried out with the real world production data. The results of KARGM model are compared to the existing discrete univariate grey prediction models, including ARGM, NDGM, DGM, and NGBMOP, and it is shown that the KARGM outperforms the other four models.

1. Introduction

The idea of the “Grey Box” modelling is trying to combine the advantages of the “White Box” and the “Black Box.” Deng [1] has pioneered the Grey System Theory based on this idea. The grey prediction models play an important role in the Grey System Theory, and because of their effectiveness in time series prediction the grey prediction models have been widely adopted [25].

Over three decades of development, many new grey prediction models have been put forward, such as the FGM [6], DGMD [7], NGM [8], and SAGM [9]. Along with these new models, some novel methodologies have also been proposed, and the discrete modelling technique is one of the most efficient methods to build the grey prediction models. The discrete modelling technique has been developed from the research of the DGM model [10], which is based on the basic GM model. This novel technique has also been used to build the NDGM model based on the NGM [11]. And in our previous works, this technique has been extended to build the discrete GM models [12, 13]. In these works, the discrete modelling technique has been proved efficient to improve the accuracy of the grey prediction models. Some novel grey prediction models for the nonlinear sequences are developed in recent years. For the univariate regression problems, the nonlinear grey Bernoulli model (NGBM) has been proposed by Chen et al. [14], which is more flexible than the existing grey prediction models and efficient to predict various time series. The NGBM model has attracted considerable research, and some improved grey prediction models based on it have been proposed, such as the Nash NGBM [15], the NGBM with optimal parameter [16], and the optimized NGBM [17] model. As for the multivariate regression problems, the nonlinear GMC model [18] has been proved to be more efficient to predict the nonlinear series than the existing models.

In recent researches, a novel grey prediction model directly built on the original series has been proposed, which is called the DDGM model [19] and is also called the autoregressive GM model (ARGM) [20] as it is essentially in the autoregressive formulation. It is unnecessary to use the 1-AGO when building the ARGM model; thus it is very easy to use. The ARGM model has been proved to coincidence with the nonhomogeneous exponential law [19], and it has also been presented to be more efficient than the DGM model in some applications [19, 20]. However, the ARGM model is essentially a linear model; thus its applicability is limited.

In order to improve the applicability of the ARGM model, we use the kernel method to build a novel kernel based ARGM model, abbreviated as KARGM. The kernel method has been developed from the Vapnik’s Support Vector Machines (SVM) [21], and it has been proved to be very efficient to convert the classical linear models into nonlinear models in the previous researches [2224]. The researches of Vapnik’s SVM [21] are the initial works of the kernel method. But the formulation of the Vapnik’s SVM is not easy to use as it involves in a quadric problem with inequivalent constraints. Suykens and Vandewalle [25] have proposed a simplified formulation of kernel method involving a quadric problem with equivalent constraints, which can be converted into a linear system. And the formulation by Suykens and Vandewalle has been proved to be available to extend to linear models into nonlinear models as efficient as the formulation of Vapnik’s SVM [26, 27]. As for the time series regression, the recurrent LS-SVM [28] is a typical model for the nonlinear univariate time series, which can be more easily used to predict the chaotic time series than the recurrent neural networks [29]. In this work, the kernel method by Suykens and Vandewalle will be used to build the KARGM model.

The rest of this paper is organized as follows. Section 2 presents a brief overview of the existing ARGM model; Section 3 presents the modelling procedures of the KARGM; two case studies of predicting the gas well production are presented in Section 4, and conclusions are drawn in Section 5.

2. Overview of the Autoregressive GM Model

With a given time series , the autoregressive GM (ARGM) model is represented as the following linear difference equation [20]:

The parameters and can be obtained using the least squares method as follows:where

The solution of the ARGM model can be obtained using the recursive method, which is given as the following discrete function:

The discrete function (4) is used to compute the predicted values.

3. The Proposed KARGM Model

In this section, the modelling procedures of the novel kernel based autoregressive GM model, abbreviated as KARGM, will be presented.

3.1. Representation of the KARGM Model

With a given time series , the KARGM model is represented as the following difference equation:where is a nonlinear mapping, which is defined asand is a higher dimensional feature space; is a vector in .

The linear combination is a nonlinear function of . For example, if we consider a nonlinear functionand define the nonlinear mapping asand set , then we have

3.2. Parameters Estimation of the KARGM Model

Being different from the ARGM model, we cannot simply use the least squares method to estimate the parameters of the KARGM model, because it is not always computationally feasible to find the formulation of the nonlinear mapping [30]. Firstly, we consider the regularized problem as follows:where is the regularized parameter to balance the flatness of the fitting curve and the error bound, and is the 2-norm. The optimization problem (10) can be solved using the KKT conditions, which have been presented in Appendix A.

We firstly define the Lagrangian function aswhere is the Lagrangian multiplier. Then the KKT conditions can be given as

By eliminating the , and , the KKT conditions can be converted to the following linear system:whereand and dimensional identity matrix with all the diagonal elements are 1 and others are zero. The inner product can be expressed using a kernel function which satisfies the Mercer’s condition, that is,The Gaussian kernel is often employed, which is defined aswhere is the kernel parameter.

The linear system (13) can be solved within the inner product (15) expressed by the Gaussian kernel (16), and then the parameter and the Lagrangian multipliers can be obtained. The parameter can then be computed using the first equation in the KKT conditions (12).

3.3. The Solution of the KARGM Model

The KARGM model (5) can be easily solved using the recursive method as follows:

Noticing the second equation in the KKT conditions (12), which can be rewritten as , we have

Using the inner product (15), we can rewrite the nonlinear function (18) as follows:

For convenience, we note the following discrete function:

Then the solution (17) can be rewritten as

The discrete function (21) can be used to compute the predicted series.

It should be noticed that we do not need to know the expression of the nonlinear mapping , because all the computational procedures of the KARGM model only involve in the inner product , and it can be expressed by a proper kernel function.

Actually, the KARGM represents a dynamical system which contains a “White” part and a “Black” part. The “White” part is the linear recursion which is known a priori, and the “Black” part is the linear combination , which is expressed by the kernel function and finally determined by the raw data. So we can say that the KARGM model is a real “Grey” model.

3.4. Summary of Computational Steps

The computational steps of the KARGM model are summarized as follows.

Step 1. Select an appropriate kernel function and the regularized parameter in (10).

Step 2. Compute the parameters and by solving the linear system (13), and then compute the parameter using the first equation in the KKT conditions (12).

Step 3. Compute the values of the predicted series using the discrete function (21).

4. Applications

Two case studies of predicting the gas well production are carried out in this section to validate the effectiveness of the KARGM model. The monthly production data are collected from two gas wells in Sichuan, China. In order to compare the performance of the KARGM model and other existing discrete grey models, the ARGM, NDGM [11], DGM [10], and the nonlinear grey Bernoulli model with optimal parameter (NGBMOP) [16] are also applied in the case studies. The mean absolute percentage error is used as an overall measurement of accuracy of the prediction models, which is defined aswhere is the real value and is the predicted value, respectively.

4.1. Case Study 1

The raw data used in case study 1 are collected from the gas well B51 in Sichuan, China. Twenty points of monthly gas production () are listed in Table 1. The first 15 points are used to build the models, and the last 5 points are used for testing. The Gaussian kernel (16) is used to build the KARGM model.

Table 1: Monthly gas production data of the B51 in Sichuan, China.

The kernel parameter and the regularized parameter are tuned using the cross validation, which is widely used in the other kernel methods, such as the Partially Linear LS-SVM [31]. The 5-fold cross validation is employed in this case. The MAPE of validation is used to assess the chosen kernel parameter and the regularized parameter. A brief summary of the 5-fold cross validation has been described in Appendix B. The results of the MAPE for each pair of and are plotted in Figure 1. It can be seen that the minimum MAPE is found at the point and ; thus this pair will be used to build the KARGM in this case.

Figure 1: The MAPE of cross validation of KARGM with different and using the 5-fold cross validation in Case Study 1.

The predicted values by the KARGM, ARGM, NDGM, DGM, and NGBMOP are listed in Table 2, along with the absolute percentage error of each point and the MAPE of each model. The results are also plotted in Figure 2.

Table 2: Predicted values of monthly gas production of B51 by the ARGM, NDGM, DGM, NGBMOP, and KARGM.
Figure 2: Prediction results of the monthly gas production of B51 by the prediction models. The raw data are plotted using the mark “+” and the predicted values are plotted using the solid line and the mark “o”. (a) ARGM; (b) NDGM; (c) DGM; (d) NGBMOP; (e) KARGM.

It can be seen in Table 2 that the minimum MAPE for fitting of ARGM, NDGM, DGM, and NBGMOP is more than three times larger than that of KARGM. The MAPE for prediction of KARGM is much smaller than the NDGM, DGM, and NBGMOP, and it is slightly larger than that of the ARGM. It is shown in Figure 2 that the predicted values of KARGM are quite close to the real values, and these values are presented to follow the overall trend of the monthly production of B51; but the distances between the predicted values of the other four models are very large, and they have failed in catching the overall trend of the gas production, which indicates that the modelling accuracy of these four models is not acceptable. In summary, the KARGM performs best in the presented models.

4.2. Case Study 2

The raw data used in case study 2 are collected from the gas well B41 in Sichuan, China. Twenty points of monthly gas production () are listed in Table 3. The first 15 points are used to build the models, and the last 5 points are used for testing. The Gaussian kernel (16) is also used to build the KARGM model.

Table 3: Monthly gas production data of the B41 in Sichuan, China.

In this case, the kernel parameter and the regularized parameter are tuned using 5-fold cross validation, and the results of the MAPE for each pair of and are plotted in Figure 4. The minimum MAPE is found at the point and ; thus this pair will be used to build the KARGM in this case.

The predicted values by the KARGM, ARGM, NDGM, DGM, and NBGMOP are listed in Table 4, along with the absolute percentage error of each point and the MAPE of each model. The results are also plotted in Figure 3.

Table 4: Predicted values of monthly gas production of B41 by the ARGM, NDGM, DGM, NGBMOP, and KARGM.
Figure 3: Prediction results of the monthly gas production of B41 by the prediction models. The raw data are plotted using the mark “+” and the predicted values are plotted using the solid line and the mark “o”. (a) ARGM; (b) NDGM; (c) DGM; (d) NGBMOP; (e) KARGM.
Figure 4: The MAPE of cross validation of KARGM with different and using the 5-fold cross validation in Case Study 2.

It can be seen in Table 4 that the minimum MAPE for fitting of ARGM, NDGM, DGM, and NBGMOP is more than two times larger than that of KARGM. The minimum MAPE for prediction of ARGM, NDGM, DGM, and NBGMOP is more than three times larger than that of the KARGM. In Figure 3 it is also shown that the predicted values of KARGM are very close to the real values and the KARGM can accurately catch the overall trend of the monthly production of B41; but the distances between the predicted values of the other four models are still very large. In this case study it is clearly that the KARGM has be best performance.

5. Conclusions

In this paper, a novel kernel based grey model (KARGM) has been proposed, and its effectiveness has been assessed in the two case studies of the gas well production forecasting in comparison to the existing discrete grey models, including the ARGM, NDGM, DGM, and NBGMOP. The results of the case studies have shown that the novel KARGM model outperformed the other three discrete grey models.

Essentially, the existing ARGM, NDGM, and DGM are linear model, and their solutions are all exponential functions. And this is also the reason why they only produce simple curves in the case studies, which cannot be used to express complex time series. The NBGMOP is a nonlinear model, but its applicability is still limited according to the results shown in the case studies. On the other hand, the KARGM model contains a general nonlinear function , which is expressed by a proper kernel function as shown in (19) and determined by the raw data. Thus the KARGM model can be more effective to deal with complex time series.

What is more, this paper has illustrated a new way of building the discrete grey model using the kernel method and also proved the effectiveness of this new methodology. Thus, the relative researches can be carried out based on this research in the future.

Appendix

A. The KKT Conditions for Equality Constrained Convex Minimization

Consider the equality constrained optimization problemwhere the objective function and the constraint functions are supposed to be continuously differentiable at point . One defines the Lagrangian function aswhere the are the Lagrangian multipliers. The point and the Lagrangian multipliers must satisfy the KKT conditions if the point is a local minimum. In this case, the KarushCKuhnCTucker (KKT) conditions can be interpreted using the Lagrangian function asThe KKT conditions are sufficient and necessary to the optimization problem (A.1) if the objective function is convex and the constrained functions are linear functions. In this case the local minimum which satisfies the KKT conditions is a global minimum; that is, the solution of the minimization problem (A.1) is equivalent to the solution of the KKT conditions (A.3). (More details of the KKT conditions can be seen in [32], pages: 243–245.)

B. A Brief Summary of the 5-Fold Cross Validation Used in the Case Studies

With a given time series , the 5-fold cross validation can be described as follows.

Step 1. Divide the original time series into 5 subsets randomly, and mark them as .

Step 2. Build the KARGM model using four subsets and the rest subset is used for validation. (For example, the first time, the KARGM model is built on the subset , and the subset is used for validation. This procedure will be repeated for 5 times, and all the subset will be used for validation.) Compute the MAPE for all the validation procedures.

Step 3. For and , repeat Step and store the values of the MAPE with all the pairs of and .

Step 4. Output the values of optimal and corresponding to the minimum MAPE.

Competing Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the Doctoral Research Foundation of Southwest University of Science and Technology (no. 16zx7140).

References

  1. J.-L. Deng, “Control problems of grey systems,” Systems & Control Letters, vol. 1, no. 5, pp. 288–294, 1982. View at Publisher · View at Google Scholar · View at Scopus
  2. X. Ma, D. Luo, X.-F. Ding, and J.-J. Zhou, “An algorithm based on the GM(1,1) model on increasing oil production of measures operation for a single well,” in Proceedings of the 24th IEEE International Conference on Grey Systems and Intelligent Services (GSIS '13), pp. 158–160, Macao, China, November 2013. View at Publisher · View at Google Scholar · View at Scopus
  3. X. Ma and Z.-B. Liu, “Predicting the cumulative oil field production using the novel grey ENGM model,” Journal of Computational and Theoretical Nanoscience, vol. 13, no. 1, pp. 89–95, 2016. View at Publisher · View at Google Scholar
  4. D. Chenyan, Z. Songtao, and D. Julong, “Numerical mapping in DNA sequences and analysis of the genetic information by GM (1, N),” Journal of Grey System, vol. 24, no. 3, 2012. View at Google Scholar
  5. Q.-F. Li, Y.-G. Dang, and Z.-X. Wang, “Analysis of the regional coordination development systems based on GRA and GM(1,N),” Journal of Grey System, vol. 24, no. 1, pp. 95–100, 2012. View at Google Scholar · View at Scopus
  6. T. L. Tien, “A new grey prediction model FGM (1, 1),” Mathematical and Computer Modeling, vol. 49, no. 7-8, pp. 1416–1426, 2009. View at Google Scholar
  7. C.-K. Chen and T.-L. Tien, “The indirect measurement of tensile strength by the deterministic grey dynamic model DGDM(1, 1, 1),” International Journal of Systems Science, vol. 28, no. 7, pp. 683–690, 1997. View at Publisher · View at Google Scholar · View at Scopus
  8. J. Cui, S.-F. Liu, B. Zeng, and N.-M. Xie, “A novel grey forecasting model and its optimization,” Applied Mathematical Modelling, vol. 37, no. 6, pp. 4399–4406, 2013. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  9. D. Q. Truong and K. K. Ahn, “An accurate signal estimator using a novel smart adaptive grey model SAGM(1,1),” Expert Systems with Applications, vol. 39, no. 9, pp. 7611–7620, 2012. View at Publisher · View at Google Scholar · View at Scopus
  10. N.-M. Xie and S.-F. Liu, “Discrete grey forecasting model and its optimization,” Applied Mathematical Modelling, vol. 33, no. 2, pp. 1173–1186, 2009. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  11. N.-M. Xie, S.-F. Liu, Y.-J. Yang, and C.-Q. Yuan, “On novel grey forecasting model based on non-homogeneous index sequence,” Applied Mathematical Modelling, vol. 37, no. 7, pp. 5059–5068, 2013. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  12. X. Ma and Z. Liu, “Predicting the oil field production using the novel discrete GM(1,N) model,” Journal of Grey System, vol. 27, no. 4, pp. 63–73, 2015. View at Google Scholar · View at Scopus
  13. X. Ma and Z.-B. Liu, “Research on the novel recursive discrete multivariate grey prediction model and its applications,” Applied Mathematical Modelling, vol. 40, no. 7-8, pp. 4876–4890, 2016. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  14. C.-I. Chen, H. L. Chen, and S.-P. Chen, “Forecasting of foreign exchange rates of Taiwan's major trading partners by novel nonlinear Grey Bernoulli model NGBM(1, 1),” Communications in Nonlinear Science and Numerical Simulation, vol. 13, no. 6, pp. 1194–1204, 2008. View at Publisher · View at Google Scholar · View at Scopus
  15. H.-T. Pao, H.-C. Fu, and C.-L. Tseng, “Forecasting Taiwans major stock indices by the Nash nonlinear grey Bernoulli model,” Energy, vol. 40, no. 1, pp. 400–409, 2012. View at Publisher · View at Google Scholar
  16. H.-T. Pao, H.-C. Fu, and C.-L. Tseng, “Forecasting of CO2 emissions, energy consumption and economic growth in China using an improved grey model,” Energy, vol. 40, no. 1, pp. 400–409, 2012. View at Publisher · View at Google Scholar · View at Scopus
  17. Z.-X. Wang, K. W. Hipel, Q. Wang, and S.-W. He, “An optimized NGBM(1,1) model for forecasting the qualified discharge rate of industrial wastewater in China,” Applied Mathematical Modelling, vol. 35, no. 12, pp. 5524–5532, 2011. View at Publisher · View at Google Scholar · View at Scopus
  18. Z.-X. Wang, “Nonlinear grey prediction model with convolution integral NGMC (1, n) and its application to the forecasting of China's industrial SO2 emissions,” Journal of Applied Mathematics, vol. 2014, Article ID 580161, 9 pages, 2014. View at Publisher · View at Google Scholar · View at Scopus
  19. B. Zeng and S.-F. Liu, “Direct modeling approach of DGM (1, 1) with approximate non-homogeneous exponential sequence,” System Engineering Theory and Practice, vol. 31, no. 2, pp. 297–301, 2011. View at Google Scholar · View at Scopus
  20. L. Wu, S. Liu, H. Chen, and N. Zhang, “Using a novel grey system model to forecast natural gas consumption in China,” Mathematical Problems in Engineering, vol. 2015, Article ID 686501, 7 pages, 2015. View at Publisher · View at Google Scholar · View at Scopus
  21. V. N. Vapnik, Statistical Learning Theory, Wiley-Interscience, 1998. View at MathSciNet
  22. B. Schölkopf, A. Smola, and K. Müller, “Kernel principal component analysis,” in Proceedings of the 7th International Conference on Artificial Neural Networks (ICANN '97), pp. 583–588, Springer, Lausanne, Switzerland, 1997.
  23. B. Scholkopft and K.-R. Mullert, “Fisher discriminant analysis with kernels,” in Proceedings of the Neural Networks for Signal Processing IX, vol. 1, pp. 41–48, August 1999.
  24. M. E. Tipping, “Sparse Bayesian learning and the relevance vector machine,” The Journal of Machine Learning Research, vol. 1, no. 3, pp. 211–244, 2001. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  25. J. A. K. Suykens and J. Vandewalle, “Least squares support vector machine classifiers,” Neural Processing Letters, vol. 9, no. 3, pp. 293–300, 1999. View at Publisher · View at Google Scholar · View at Scopus
  26. T. Van Gestel, J. A. K. Suykens, G. Lanckriet, A. Lambrechts, B. De Moor, and J. Vandewalle, “Bayesian framework for least-squares support vector machine classifiers, gaussian processes, and kernel fisher discriminant analysis,” Neural Computation, vol. 14, no. 5, pp. 1115–1147, 2002. View at Publisher · View at Google Scholar · View at Scopus
  27. J. A. K. Suykens, T. Van Gestel, J. Vandewalle, and B. De Moor, “A support vector machine formulation to PCA analysis and its kernel version,” IEEE Transactions on Neural Networks, vol. 14, no. 2, pp. 447–450, 2003. View at Publisher · View at Google Scholar · View at Scopus
  28. J. A. K. Suykens and J. Vandewalle, “Recurrent least squares support vector machines,” IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 47, no. 7, pp. 1109–1114, 2000. View at Publisher · View at Google Scholar · View at Scopus
  29. L. R. Medsker and L. C. Jain, Recurrent Neural Networks: Design and Applications, CRC Press, Boca Raton, Fla, USA, 2001.
  30. A. J. Smola and B. Schölkopf, “A tutorial on support vector regression,” Statistics and Computing, vol. 14, no. 3, pp. 199–222, 2004. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  31. M. Espinoza, J. A. K. Suykens, and B. De Moor, “Partially linear models and least squares support vector machines,” in Proceedings of the 43rd IEEE Conference on Decision and Control (CDC '04), vol. 4, pp. 3388–3393, IEEE, December 2004. View at Scopus
  32. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004. View at Publisher · View at Google Scholar · View at MathSciNet