Crude Oil Price Prediction Based on a Dynamic Correcting Support Vector Regression Machine

Abstract

A new accurate method on predicting crude oil price is presented, which is based on ε-support vector regression (ε-SVR) machine with dynamic correction factor correcting forecasting errors. We also propose the hybrid RNA genetic algorithm (HRGA) with the position displacement idea of bare bones particle swarm optimization (PSO) changing the mutation operator. The validity of the algorithm is tested by using three benchmark functions. From the comparison of the results obtained by using HRGA and standard RNA genetic algorithm (RGA), respectively, the accuracy of HRGA is much better than that of RGA. In the end, to make the forecasting result more accurate, the HRGA is applied to the optimize parameters of ε-SVR. The predicting result is very good. The method proposed in this paper can be easily used to predict crude oil price in our life.

1. Introduction

In recent years, crude oil prices have experienced four jumps and two slumps. The fluctuation of crude oil price adds more changes to the development of world economy. Grasping the change of oil price can provide guidance for economic development [1]. Therefore, it is very important to predict the crude oil price accurately.

The predicting methods can be divided into two aspects. One is from the qualitative angle [2]; the other is from quantitative angle, such as econometric model and statistical model [3, 4]. And the latter method is adopted by most scholars. But it is a difficult job to predict crude oil price, since the price is nonlinear and nonstationary time series [5]. The traditional predicting methods such as 𝐴𝑅(𝑝) model, 𝑀𝐴(𝑞) model, and 𝐴𝑅𝑀𝐴(𝑝,𝑞) model, base on linear model. They are only suitable for linear prediction and cannot be applied to model and predict nonlinear time series [6]. Wang got the predicting model by using time series and artificial neural network in 2005 [7], Xie proposed a new method for crude oil price forecasting based on support vector machine (SVM) in 2006 [8], Mohammad proposed a hybrid artificial intelligence model for crude oil price forecasting by means of feed-forward neural networks and genetic algorithm in 2007 [9], and Guo proposed a hybrid time series model on the base of GMTD model in 2010 [10]. The experimental results tell us that the prediction accuracy of these methods is better than traditional models. But the results is still existing biggish errors especially when the crude oil price is fluctuating violently.

Neural network technique provides a favorable tool for nonlinear time series forecasting. But the predictive ability of conventional neural network is low, because of the problems such as the local minimum, over learning, and the lacking of theoretical direction for selecting the hidden layer nodes. The SVM was proposed in the 1990s [11]; it can get the optimal results on the basis of the current information. The basic idea of SVM is that it fits the sample capacity of functions on the basis of regulating the upper bound of the minimum VC dimension, which also means the numbers of support vector. Compared with neural network [12, 13], SVM has strong generalization ability of learning small samples and with the inferior dependence on quantity. But the prediction performance of SVM is very sensible to parameter selection. On the other hand, the research on parameter optimization of SVM is very few at the moment. The parameters are usually determined on experience or trial method. In this way, if the parameters are not suitably chosen, the SVM will lead to poor prediction performance. So, it is important to find one good method to get the optimal parameters of SVM.

In this paper, an 𝜀-support vector regression machine with dynamic correction factor is proposed. And a novel hybrid RNA genetic algorithm (HRGA) is proposed to obtain the optimal parameters for a SVM. The HRGA is from the development of biological science and technology; the structure and information of RNA molecular are known profoundly. To improve the optimal performance of genetic algorithm, one genetic algorithm which bases on coding and biological molecular operation has been widely concerned [14]. This method improves the search efficiency and optimization performance through coding the individuals into biological molecules by use of bases [15, 16]. The appropriate mutation operator can improve the population diversity and prevent premature. While the mutation operator of classical RNA genetic algorithm (RGA) is fixed, so we need to find a suitable method to determine the mutation operator. In 2003, Kennedy did some improvement on particle swarm optimization (PSO) and proposed the bare bones particle swarm algorithm [17].

In the proposed HRGA, the position displacement idea of bare bones PSO is applied to change the mutation operator. The nucleotide base encoding, RNA recoding operation, and protein folding operation are reserved in the new algorithm. Thus, the strong global search capability is kept. At the same time, to make sure of the directivity of local searching, the optimal experience of the whole population and the historical experience of individuals are used. The convergence speed and solution precision are improved. Furthermore, to test the validity of HRGA, three benchmark functions are adopted. The mean value of optimum of HRGA is smaller than that of traditional RNA genetic algorithm.

Once the support vector regression machine is designed optimally, it can be used to predict crude oil price. Dynamic correction factor is brought in to improve the predictive effect and can strengthen the robustness of systems. In order to test the performance of the proposed predicting method, we provided the predicting results by using a back propagation neural network and a traditional support vector regression machine which are also improved with dynamic correction factor [7, 8]. The results show that our predicting method obtains greater accuracy than that of the other two in this paper.

The paper is organized as follows. Section 2 discusses the support vector regression machine with dynamic correction factor. Section 3 presents HRGA based on bare bones PSO, and some testing examples are applied to verify the effectiveness of the algorithm. Section 4 applies the dynamic correcting 𝜀-SVR to predict the crude oil price. Section 5 concludes the paper.

2. Support Vector Regression Machine with Dynamic Correction Factor

Consider the training sample set (𝑥_𝑖,𝑦_𝑖), 𝑖=1,2,…,𝑛, 𝑥_𝑖∈𝑅^𝑛, as the input variable and 𝑦_𝑖∈𝑅 as the output variable.

The basic idea of SVM is to find a nonlinear mapping 𝜙 from input space to output space [18–20]. Data 𝑥 is mapped to a high-dimensional characteristic space 𝐹 on the basis of the nonlinear mapping. The estimating function of linear regression in characteristic space 𝐹 is as follows: 𝑓(𝑥)=𝜔×𝜑(𝑥)+𝑏,𝜙∶𝑅^𝑛⟶𝐹,𝜔∈𝐹, (1) where 𝑏 denotes threshold value.

Function approximation problem is equal to the following function: 𝑅_reg𝑓=𝑅_emp𝑓+𝜆‖𝜔‖²=^𝑠_𝑖=1𝐶𝑒_𝑖+𝜆‖𝜔‖², (2) where 𝑅_reg(𝑓) denotes the objective function, 𝑅_emp(𝑓) denotes the empirical risk function, 𝑠 denotes the sample quantity, 𝜆 denotes adjusting constant, and 𝐶 denotes the error penalty factor. ‖𝜔‖² reflects the complexity of 𝑓 in the high-dimensional characteristic space.

Since linear 𝜀 insensitive loss function has better sparsity, we can get the following loss function: 𝑦−𝑓(𝑥)_𝜀=max0,𝑦−𝑓(𝑥)−𝜀. (3)

The empirical risk function is as follows: 𝑅^𝜀_emp𝑓=1𝑛^𝑛_𝑖=1𝑦−𝑓(𝑥)_𝜀. (4)

According to the statistical theory, we bring in two groups of nonnegative slack variable {𝜉_𝑖}^𝑛_𝑖=1 and {𝜉^∗_𝑖}^𝑛_𝑖=1. Then, the question can be converted to the following nonlinear 𝜀-support vector regression machine (𝜀-SVR) problem: min_{𝜉𝑖,𝜉∗𝑖}12‖𝜔‖²+𝐶^𝑛_𝑖=1𝜉_𝑖+𝜉^∗_𝑖,𝑦_𝑖−𝜔⋅𝜙(𝑥)−𝑏≤𝜀+𝜉^∗_𝑖,𝜔⋅𝜙(𝑥)+𝑏−𝑦_𝑖≤𝜀+𝜉_𝑖,𝜉_𝑖,𝜉^∗_𝑖≥0, (5) where 𝜀 denotes the insensitive loss function. 𝐶 is used to balance the complex item and the training error of the model.

We bring into Lagrange multipliers 𝛼_𝑖 and 𝛼^∗_𝑖, then the convex quadratic programming problem above can be changed into the below dual problem: max_{𝛼𝑖,𝛼∗𝑖}−12^𝑛_𝑖,𝑗=1𝛼^∗_𝑖−𝛼_𝑖𝛼^∗_𝑗−𝛼_𝑗𝐾𝑋_𝑖,𝑋_𝑗+^𝑛_𝑖=1𝛼^∗_𝑖𝑦_𝑖−𝜀−^𝑛_𝑖=1𝛼_𝑖𝑦_𝑖−𝜀_𝑛𝑖=1,𝛼_𝑖−𝛼^∗_𝑖=0,0≤𝛼_𝑖,𝛼^∗_𝑖≤𝐶,𝑖=1,2,…,𝑛, (6) where 𝐾(𝑋_𝑖,𝑋_𝑗) denotes the inner product kernel satisfying Mercer theorem.

We can get the 𝜀-𝑆𝑉𝑅 function through solving the above dual problem: 𝑓(𝑥)=^𝑛_𝑖=1𝛼_𝑖−𝛼^∗_𝑖𝐾𝑋_𝑖,𝑋+𝑏. (7)

When 𝜀-𝑆𝑉𝑅 is used on prediction, it may have a certain error since the data fluctuates violently such as the crude oil price. To reduce the error in some certain as possible as we can, we bring in the dynamic correction factor 𝜀^∗. The main idea of the dynamic correction factor is that we use the error of back step with multiplying 𝜀^∗ to revise the current predicting results. Thus, we can reduce the current predicting error. The dynamic correcting SVR can be defined as follows: 𝑌_𝑑(𝑖+1)=𝑌₁(𝑖+1)+𝜀^∗𝑌(𝑖)−𝑌_𝑑(𝑖), (8) where 𝑌 denotes the real results, 𝑌_𝑑 denotes the final prediction results, 𝑌₁ denotes the initial predicting results, 𝜀^∗ denotes the dynamic correction factor, and 𝑖 denotes the prediction steps.

In order to make the predicting results more accurate, the optimal value of 𝜀^∗ and the parameters of 𝜀-𝑆𝑉𝑅 involving 𝐶,𝛿 (the variable in gauss kernel function) should be designed (in (8)). To this end, an HRGA is studied below to optimize the following problem: min_(𝜀∗)^𝑛_{,𝐶,𝜎𝑖}𝑌_𝑑(𝑖)−𝑌(𝑖)². (9)

3. HRGA Based on Bare Bones PSO

Assuming that population size is 𝑁, the dimension of particle is 𝑚. The position of particle 𝑖 on generation 𝑡 is 𝑋_𝑖(𝑡)=(𝑥_𝑖1(𝑡),…,𝑥_𝑖𝑗(𝑡),…,𝑥_𝑖𝑚(𝑡)),𝑖=1,2,…,𝑁. The speed of particle 𝑖 on generation 𝑡 is 𝑉_𝑖(𝑡)=(𝑣_𝑖1(𝑡),…,𝑣_𝑖𝑗(𝑡),…,𝑣_𝑖𝑚(𝑡)). The historic optimal value of individuals is 𝑃𝐵𝑒𝑠𝑡_𝑖(𝑡)=(𝑝𝑏𝑒𝑠𝑡_𝑖1(𝑡),…,𝑝𝑏𝑒𝑠𝑡_𝑖𝑗(𝑡),…,𝑝𝑏𝑒𝑠𝑡_𝑖𝑚(𝑡)).

Let the global optimal value be 𝐺𝐵𝑒𝑠𝑡(𝑡)=(𝑔𝑏𝑒𝑠𝑡₁(𝑡),…,𝑔𝑏𝑒𝑠𝑡_𝑗(𝑡),…,𝑔𝑏𝑒𝑠𝑡_𝑚(𝑡)).

As to standard particle swarm, the position and speed are updated as 𝑣_𝑖𝑗(𝑡+1)=𝑤𝑣_𝑖𝑗(𝑡)+𝑐₁⋅𝑟_1𝑗⋅𝑝𝑏𝑒𝑠𝑡_𝑖𝑗(𝑡)−𝑥_𝑖𝑗(𝑡)+𝑐₂⋅𝑟_2𝑗⋅𝑔𝑏𝑒𝑠𝑡_𝑗−𝑥_𝑖𝑗(𝑡),𝑥_𝑖𝑗(𝑡+1)=𝑥_𝑖𝑗(𝑡)+𝑣_𝑖𝑗(𝑡+1), (10) where 𝜔 denotes the inertia weight [21], 𝑐₁ and 𝑐₂ denote the accelerating operators, and𝑟_1𝑗 and 𝑟_2𝑗 are uniform distributed random numbers in [0,1].

In the bare bones particle swarm optimization (PSO), (10) is replaced by (11) as the evolution equation of particle swarm algorithm: 𝑥_𝑖𝑗(𝑡+1)=𝑁0.5𝑝𝑏𝑒𝑠𝑡_𝑖𝑗(𝑡)+𝑔𝑏𝑒𝑠𝑡_𝑗(𝑡×)𝑝𝑏𝑒𝑠𝑡_𝑖𝑗(𝑡)−𝑔𝑏𝑒𝑠𝑡_𝑗(𝑡). (11)

The position of particle is some random numbers which are gotten from the Gauss distribution. The distribution has the mean value of (𝑝𝑏𝑒𝑠𝑡_𝑖𝑗(𝑡)+𝑔𝑏𝑒𝑠𝑡_𝑗(𝑡))/2 and the standard deviation of |𝑝𝑏𝑒𝑠𝑡_𝑖𝑗(𝑡)−𝑔𝑏𝑒𝑠𝑡_𝑗(𝑡)|.

RNA genetic algorithm is on the basis of base coding and biological molecules operation. Since in the biological molecule, every three bases compose one amino acid. In other words, the bases’ length of individuals must be divided exactly by 3. When RNA recoding and protein folding [22], to reduce calculation and to control population size, we assume that the protein folding operation only occurs on the individuals without RNA recoding. Then the most important work is to change the mutation probability [23, 24].

Angeline told us that the essence of particle swarm’s position updating was one mutation operation in 1998 [25]. Traditional RNA genetic algorithm mutates as the fixed mutation probability, and the mutation is random with one direction. However HRGA can reflect the historic information of individuals and the sharing information of the population. HRGA can make every individual do directional mutation and improve search efficiency. Moreover, HRGA ensures the strong global search capability, since it does not change the selection and crossover operator.

The procedure of HRGA based on bare bones particle swarm algorithm to optimize the 𝜀-𝑆𝑉𝑅 parameters and the dynamic correction factor is as follows.

Step 1. Get one group of 𝜀-𝑆𝑉𝑅 parameters, and the dynamic correction factor randomly, code every parameter, and get the initial RNA population with 𝑁 individuals, crossover probability 𝑃_𝑐, and mutation probability 𝑃_𝑚. Assign values for every 𝑃𝐵𝑒𝑠𝑡_𝑖 (individual’s historic optimal solution) and 𝐺𝐵𝑒𝑠𝑡 (population’s global optimal solution).

Step 2. Compute its error function and get the fitness function. Comparing it with corresponding fitness value of 𝑃𝐵𝑒𝑠𝑡_𝑖 and 𝐺𝐵𝑒𝑠𝑡, then update 𝑃𝐵𝑒𝑠𝑡_𝑖 and 𝐺𝐵𝑒𝑠𝑡.

Step 3. Execute the selection operation. Get current generation through coping 𝑁 individuals from the initial or the last generation.

Step 4. Decide whether the value meets the RNA recoding condition or not. If 𝑌, recode RNA, then go to Step 6. If 𝑁, go to Step 5.

Step 5. Decide meet the protein mutual folding condition or not. If 𝑌, execute the protein mutual folding operation. If 𝑁, execute the protein own folding operation.

Step 6. Execute the mutation operation as (11) for all the crossover individuals, on the basis of the 𝑃𝐵𝑒𝑠𝑡_𝑖 and 𝐺𝐵𝑒𝑠𝑡, which have been gotten from Step 2.

Step 7. Repeat Step 2 to Step 6 until the training target meets the condition. At last, we get the optimal parameters of 𝜀-𝑆𝑉𝑅 and the dynamic correction factor.

The flowchart of HRGA to optimize the 𝜀-𝑆𝑉𝑅 parameters and the dynamic correction factor is shown in Figure 1.

Figure 1 

The flowchart of HRGA.

3.1. HRGA Testing

Three classical benchmark functions shown in Table 1 are used to test the property of HRGA.

Table 1 

Benchmark functions.

In addition, among the three functions, Sphere is unimodal function, and the other two are multimodal function.

With the population size 𝑁=50, and other parameters determined by multiple test for each function. Each function is tested by HRGA and standard RGA in different dimensions. Each experience is carried on 100 times. Record the mean value of target function’s optimum (shown in (12)). The result is displayed in Table 2: 1MVO=𝑁^𝑁_𝑖=1𝑓_𝑖^′(𝑥). (12)

Table 2 

Testing results of  HRGA and standard RGA.

In this equation, MVO denotes the mean value of target function’s optimum; 𝑓_𝑖^′(𝑥) denotes the optimum of benchmark functions in every experiment.

As to the experimental results, with different dimensions having the same iterative times, the mean value of optimum of HRGA is smaller than that of RGA for the three benchmark functions. The average performance of HRGA is closer to the optimum. We can increase the mutation probability appropriately and enhance the convergence speed, since the mutation operator of HRGA has directional local search.

4. Crude Oil Price Prediction Based on a Dynamic Correcting 𝜀-SVR

In this paper, we get the crude oil price from the US Energy Information Administration Web [26]. Since the oil price fluctuates violently, in order to facilitate the processing and decrease the error, we adopt the Cushing, OK WTI Spot Price FOB (dollars per barrel)monthly from January 1986 to now. We take the one hundred data from January 1986 to April 1994 as the test sample. And give the next 20-month dynamic predicting data from May 1994 to December 1995. The relative error of forecasting is shown in Table 2. The prediction effect figure of HRGA and 𝜀-𝑆𝑉𝑅 with dynamic correction factor is shown in Figure 2. We use Gauss function as the kernel function of 𝜀-𝑆𝑉𝑅, which is given as follows: 𝑘𝑥,𝑦=exp−𝑥−𝑦²2⋅𝜎². (13)

Figure 2 

The prediction curve of HRGA-𝜀-𝑆𝑉𝑅.

Parameter setting of HRGA-𝜀-𝑆𝑉𝑅 is with population size being 100, maximum evolution generation being 150, coding length of 𝐶 being 9, coding length of 𝜀 being 8, coding length of 𝜎 being 13, coding length of 𝜀^∗ being 8, 𝑃_𝑐 being 0.8, and 𝑃_𝑚 being 0.1.

The optimization interval is set to be 1≤𝐶≤10000,0.0001≤𝜀≤0.1,0.01≤𝜎≤500,0.5≤𝜀^∗≤2. (14)

When analyzing the results, we define the evaluation index: 𝐸_𝑟=𝑥_𝑖−𝑦_𝑖𝑥_𝑖×100%,𝜎=1𝑛^𝑛_𝑖=1𝑥_𝑖−𝑦_𝑖𝑥_𝑖². (15)

The forecasting error analysis results are shown in Figure 3. In this figure, SVM refers to 𝜀-𝑆𝑉𝑅. The BP neural network and 𝜀-𝑆𝑉𝑅 are with dynamic correction factor which differs them to the traditional method. From Figure 2, we can know that the prediction result is very close to the real value. The HRGA-𝜀-SVR can be used to predict the crude oil price. Table 3 tells us the WTI crude oil price predicting relative errors of twenty months. Among the three methods in twenty months, the biggest absolute value of relative error of HRGA-𝜀-𝑆𝑉𝑅 is the smallest, which is 7.35%, and the smallest root-mean-square of relative error is 3.87%. As to Figure 3, the fluctuation range of HRGA-𝜀-𝑆𝑉𝑅 is smaller than those of the other two methods obviously. This means that HRGA-𝜀-𝑆𝑉𝑅 is the best one among the three methods.

Table 3 

Analysis results of forecasting error.

Figure 3 

Errors analysis with different models.

5. Conclusions

In this paper, we have presented a novel method on predicting crude oil price. This method bases on an 𝜀-support vector regression machine with dynamic correction factor correcting predicting errors. We also proposed the HRGA, with the position displacement idea of bare bones PSO changing the mutation operator, to optimize the parameters in an 𝜀-SVR. The predicting result of crude oil price shows the validity of the proposed method. Thus, the 𝜀-SVR model can also be applied to predict tendency in other practical areas.

Acknowledgments

The research was partially supported by Grant no. 60974039 from the National Science Foundation of China and by Grant no. ZR2011FM002 from the Natural Science Foundation of Shandong Province.