Abstract
With the reform of electric power system, major progress has been made in the construction of the electricity market. Electricity prices are a key influencing factor in the electricity market, and each participant trades electricity based on the price of electricity. Therefore, improving the accuracy of electricity price forecasts is important for every player in the electricity market. Prediction using single-layer neural networks has limited accuracy. Due to the high accuracy of machine learning in forecasting, the method of deep belief network is used to predict the price of electricity in the future. Real data from the U.S. PJM electricity market are used for simulation and compared with the prediction models of other neural networks. The results show that the prediction accuracy of the deep belief network model is higher, and the use of the deep belief network can provide an effective method for China’s electricity sales companies to predict electricity prices.
1. Introduction
In recent years, significant progress has been made in the construction of the electricity market. Electricity prices are an important factor in the electricity market, which can ensure the stable operation of the market, and the prediction of electricity prices has gradually become the focus of scholars’ attention [1, 2]. Electricity prices are a key influencing factor in the electricity market, and each participant trades electricity based on the price of electricity. Therefore, improving the accuracy of electricity price forecasts is important for every player in the electricity market [3, 4]. Therefore, further research on electricity price forecasting is urgently needed.
In 2011, Abbas Khosravi proposed an interval prediction method of Lower Upper Bound Estimation (LUBE) [5]. This method adjusts the single output of NN into two outputs and directly outputs the upper and lower bounds of the prediction interval. Prediction Interval Coverage Probability (PICP), Prediction Interval Normalized Average Width (PINAW), and Accumulated Width Deviation (AWD) are evaluation indicators. By assigning appropriate weight coefficients to each evaluation indicator, construct a comprehensive objective function, and use the comprehensive objective function to minimize network parameters to achieve interval prediction.
Many researchers proposed different types of NN for interval prediction and optimized the network parameters of NN with a heuristic algorithm. The authors in [6] proposed an interval prediction method combining adaptive fuzzy neural reasoning and simulated an annealing algorithm. The authors in [7] combine multilayer perceptron with Particle Swarm Optimization (PSO) algorithm to analyze wind power and load interval forecasting. The authors in [8] combined extreme learning machine and PSO for interval prediction of wind power. The authors in [9] combine wavelet neural network and improved artificial bee colony for wind power interval prediction. The authors in [10] used support vector machine to predict the upper and lower bounds of electricity price and used PSO to optimize the hyper parameters of support vector machine. The authors in [11] proposed a method for predicting electricity price interval based on a residual neural network.
At present, there are many methods of electricity price forecasting at home and abroad, which are categorized as follows. The authors of [12] proposes to establish a time series model of the residual range of electricity prices on the existing electricity price prediction model and use the autoregressive integrated moving average model (ARIMA) to reduce the prediction error. The time series combination model is simple and easy to understand, and the calculation speed is fast, and the disadvantage is that the accuracy of the prediction is low. Given the low accuracy of time series combination models, some researchers have begun to use neural network methods for electricity price prediction. In [13, 14], the prediction model of convolutional neural network is proposed to predict the price of electricity before the day. In [15], a method was designed to first use the ensemble empirical mode decomposition (EEMD) method to decompose the historical electricity price and then use the wavelet neural network to predict the electricity price. In order to improve the prediction accuracy, the authors in [4] designed a new neural network model that combines the wavelet transform with the vector function to predict the price of electricity. Some studies have proposed to combine the above two models. In [16, 17], the model of regression neural network is used for electricity price prediction. The authors in [18, 19] proposes a combined prediction model based on EEMD, support vector machine (SVM), and autoregressive moving average model (ARMA), but the accuracy of these combination models still needs to be improved.
In order to further improve the prediction accuracy, this paper uses the prediction model of the deep belief network [20] to predict the price of electricity before the day. Electricity prices have characteristics of time series. Deep belief networks can better capture the timing characteristics of electricity price fluctuations. The wavelet transform is combined with a DBN, which decomposes the original electricity price signal, then provides a good initial value for all parameters through hierarchical pretraining, and then searches for the optimal value by supervising the fine-tuning process. Compared with other single-layer neural networks, this paper proposes that the training process of deep belief network is hierarchical. The number of iterations is large, and the prediction accuracy is higher.
1.1. Influencing Factors of Electricity Price
In the case that renewable energy does not participate in the market competition, the market electricity price is mainly affected by the supply and demand relationship, unit quotation, transmission line blockage, and other factors, which has the characteristics of nonlinear and nonstable [21]. When renewable energy to participate in market competition, due to the low cost of renewable energy generation, and output has uncertainty, intermittent, and volatility, it is more likely to cause the price of electricity fluctuations, when the electricity price forecast needs to consider not only the generation and electricity supply and demand on the impact of electricity price but also the influence of renewable energy generation on electricity price.
In the electricity market, the market trading center has disclosed the information to the market members. The information includes not only historical electricity price data but also transaction supply and demand information, electricity generation forecast, electricity consumption forecast, renewable energy generation forecast, and other information. In order to improve income and avoid price risk, market members need to decide the quotation strategy after integrating all kinds of information. Electricity prices are affected by many factors. There are correlations among the multiple factors. DBN is an inference network based on probabilistic uncertainty and provides an effective way for learning and inference of causal information. The DBN can predict the state variables on each time slice. DBN has been applied in photovoltaic power generation probability forecast [22], load forecast [23], and economic index forecast [24].
According to the expert knowledge of the main factors affecting electricity price, this paper takes wind power generation, total power generation, and total electricity consumption as the explanatory variables of electricity price. We then construct the DBN model to predict the electricity price interval by evidence inference.
2. Deep Belief Network Model
Several traditional electricity price prediction models are introduced earlier. The deep belief network (DBN) is introduced in this paper. DBNs are mainly used in feature learning, data classification, and data generation. In DBNs, restricted Boltzmann machine (RBM) is primarily used as unsupervised learning subparts of the building blocks, plus a logistic regression layer for prediction.
2.1. Restricted Boltzmann Machine
The Boltzmann machine (BM) is a type of neural network. BM is a neural network with a two-layer neuronal structure and the visible layer (VL) composed of elements, which is mainly used to input data that needs to be trained [16]. The hidden layer (HL) composed of hidden elements is mainly used to detect features. The structure of the Boltzmann machine is shown in Figure 1(a).
(a)
(b)
BM has a strong ability to learn characteristics, because of its training and learning takes a long time. To optimize its time-consuming drawbacks, Sejnowski proposed RBM on the basis of BM.
The difference between RBM and BM is that neurons of the same layer are disconnected from each other in RBM. Neurons of different layers are completely connected in both directions in RBM. The advantage of this connection is that the number of neurons per layer is uncorrelated with each other. There is no connection between the RBM and the layer, which reduces the process of information transfer and therefore reduces the time to solve.
RBM can flow in both directions, which is equivalent to increasing the number of iterations of training and ensuring the accuracy of the predictive model. This will make the accuracy of the prediction not drop too much. The RBM structure is shown in Figure 1(b).
RBM represents the probability distribution from the explicit layer to the hidden layer through the energy function, given the feature , the hidden element , its connection weight , and the offset ; its energy function can be defined.
The energy function is used to define the probability distribution of the explicit and implicit layers.
Here, is the normalized constant of the distribution function. represents the weight between each hidden element and the feature.
Here, is the weight value from the th feature to the th hidden element. is the number of cells. is the number of hidden elements.
Thus, the conditional distribution probabilities for each feature and hidden element can be derived as follows:
2.2. Deep Belief Network
Deep belief networks are stacked in cascades by multiple RPMs, and the DBN training process consists of pretraining and fine-tuning [19, 20].
The DBN training process includes hierarchical pretraining and fine-tuning. Hierarchical pretraining provides initial values for all parameters, while fine-tuning explores optimal values based on the network structure. The deep belief network pretrain process is shown in Figure 2.
The layer-by-layer training process is as follows: (1)The next RBM is fully trained using raw input data(2)Fix the weight and offset of the first RBM, and take the features extracted by the bottom RBM as the input to the top RBM; these two hidden layers can then be thought of as a new RBM and trained in the same way(3)After the second RBM has been fully trained, it is stacked on top of the first RBM, and the process is repeated to train as many RBM layers as possible(4)Repeat the above 3 steps as many times as possible(5)Finally, add a standard predictor at the top, logistic regression, and the training of predictors is called a fine-tuning process, which is designed to slightly adjust the parameters in the entire network
3. The Fine-Tuning Process of the DBN
3.1. Wavelet Decomposition
The original electricity price contains many nonlinear and nonstationary factors, which are one of the reasons for the reduction of the accuracy of predicting electricity prices. Therefore, we need to use wavelet transforms to decompose historical data into multiple frequencies to give the data better performance in terms of variance and outliers.
Wavelet transforms can be in discrete form to improve efficiency. The discrete form of the wavelet transform is shown as follows:
Here, is the mother wave. and are the two integer variables that determine the scale and translation parameters. is an indicator of discrete time. is the length of the signal .
This paper uses an algorithm based on Mallat’s fast discrete wavelet transform. The algorithm consists of a decomposition filter and a reconstruction filter. Thus, a multilevel decomposition process based on the Mallat algorithm can decompose historical electricity price data into one approximation (An) and multiple granular values (Dn), as shown in Figure 3.
3.2. DBN-Based Hierarchical Pre-Training
Traditional neural networks are very prone to get stuck in the problem of local optimization. When the neural network is a deep network, this problem will become more prominent because there are more parameters to optimize. The way to deal with local optimality problems is to initialize the parameters as much as possible.
DBNs have a better way of solving these problems. The DBN training process includes hierarchical pretraining and fine-tuning. Hierarchical pretraining provides initial values for all parameters, while fine-tuning explores optimal values based on the network structure.
Each time you pretrain an independent RBM, you can get the relevant parameters . The pretraining process is achieved by performing a random gradient rise of the RBM objective function, i.e. the log-likelihood of .
Here, is the probability of the visible vector on all hidden units. Therefore, the objective function takes the form as follows:
Here, . are the training datasets.
According to Bayesian statistical theory, the objective function (9) is maximized by the stochastic gradient ascending algorithm to generate a stable and initially good RBM. The gradient ascending algorithm shows that the parameters in the RBM are updated according to the derivative of the objective function , as shown below
Here, and are raw data driven and reconstructed data-driven probabilities, respectively.
In the paper, we use equations (4) and (5) on the training dataset. It is easy to compute EP[] of equations (9)–(11). However, the calculation of items 2 in equations (9) through (11) is much more complex because the DBN system learns the expected value of the distribution . One possible strategy is to apply alternating Gibbs sampling on any random state of the visible unit until some convergence criterion, such as the -step, is met. Therefore, the expectation of can be estimated by analysis. However, sampling strategies are time-consuming and therefore rarely enforced in real life. As a remedy, a quick learning method called contrastive divergence (CD) is proposed. This method takes two approaches to speed up the sampling process. One is to initialize the Markov chain with training samples, and the other is to obtain samples after only the Gibbs sampling step, called CD-. Experimental results show that even if , CD can do a good job of model recognition.
This paper uses CD-1 to estimate the expected value of . So, the update rules for parameters can be derived from equations (9)–(11).
Here, the superscript is the time step. is the learning rate, which is 0.9 in this paper.
3.3. The Fine-Tuning Process of Supervise
Recent studies have found that the accuracy of the prediction is the highest when the number of layers of the RBM is 4. As described in Section 2.2, the number of neurons per layer in a DBN is properly initialized based on a hierarchical pretraining method. These parameters need to be fine-tuned under supervision until the loss function of the DBN reaches a minimum. Based on the effectiveness of the BP algorithm, this paper uses the BP algorithm to deal with such tasks.
In the supervised fine-tuning process, the BP algorithm works in a top-down manner based on a certain cycle. One duty cycle means that all parameters are updated at once, which will reduce the error of prediction. Next, these errors are backpropagation by the training set, and then the DBN parameters are readjusted to the optimal state. Therefore, after repeating a certain BP cycle, the optimal DBN parameters can be obtained, which means that the training process of the deep belief network is completed.
3.4. DBN Inference
DBN inference is based on the established DBN at time , adding the sample of explanatory variables at time (inferential evidence) to infer the discrete state (cluster category) and the posterior probability of the predictor variable at time .
Wind power generation , total electricity generation , and total electricity consumption are taken as the explanatory variables of electricity price EP. Assuming that the joint probability distribution over the time trajectories has been obtained. If the sample at time is added as inference evidence, the posterior probability is obtained by DBN inference. The posterior probability of the electricity price at time is gradually obtained. The DBN evidence inference procedure is shown in Figure 4.
DBN inference has both exact inference and approximate inference. In this paper, the joint tree inference algorithm of DBN is used. Joint tree inference constructs a 1.5 DBN joint tree using nodes within two adjacent time slices in the DBN. The evidence is then entered to reasoning using the forward-backward algorithm, after the inference is finished. A predictor variable is at marginalized. The corresponding posterior probability was calculated when the predictor variable takes different discrete values.
Using the Bayesian network toolbox FullBNT-1.0.7, the specific steps to achieve exact inference are as follows. (1)Convert the DBN model to a 1.5DBN joint tree(2)Input evidence and reason by using the forward-backward algorithm(3)Calculate the posterior probability when the electricity price takes different discrete values at each time(4)The predicted average of electricity price at time is computed(5)The lower and upper bounds of the electricity price prediction interval at the time point can be obtained
4. Simulation and Analysis
This chapter describes the experimental environment and verifies the effectiveness and accuracy of the proposed model through simulation experiments.
4.1. Experimental Environment
In order to verify the superiority of the forecasting model proposed in this paper, the real data of the US PJM electricity market [12] is used for simulation and prediction.
In the paper, we select 61 days of historical electricity price data from October 1, 2018, to November 30, 2018, to predict the electricity price of December 1, 2018. We take as the sampling period and take the sample data of the first 40 days with a total of 960 h as the training sample. We take the sample data of the next 21 days with a total of 504 h as the prediction set. In this paper, the historical electricity price data is entered in the Matlab simulation platform, the scale of this data can avoid the simulation process taking too long under the premise that the accuracy is basically up to the requirements, and the sample characteristics are trained, and the 1-day electricity price prediction result is obtained by the sample set prediction of 504 h, and the effectiveness of the proposed model prediction of the electricity price is verified by comparing with the actual electricity price data, and the electricity price prediction is verified by comparing with the EEMD and GA-SVM. The prediction results of the combined electricity price prediction model such as the ARMA composite model, the GA-SVM combined model, and the ARMA-GARCH combined model illustrate the accuracy of the forecasting model in this paper. The prediction results of various models are shown in Figure 5.
4.2. Test Results and Analysis
By entering historical electricity price data on the Matlab simulation platform, the simulation results of the forecast electricity price of the US PJM electricity market as of December 1, 2018, are as follows. The number of neurons in each layer of the DBN is shown in Table 1.
The forecast electricity price data in this article is shown in Figure 6. The metric is compared to the real electricity price pair for the US PJM electricity market as of December 1, 2018. The pairing of the prediction data of the Boltzmann machine model with the real electricity price is shown in Figure 7.
The error pairs of the DBN prediction model and the Boltzmann machine prediction model are shown in Table 2.
RBM information can flow in both directions, which is equivalent to increasing the number of iterations of training and ensuring the accuracy of the predictive model. By comparing the errors of the DBN prediction model with the BM prediction model, we can see that the average error of the DBN prediction model is lower than the average error of the BM prediction model. Therefore, using RBM to predict electricity prices, the prediction accuracy does not decrease, but increases slightly. The pairing of forecast data from other models with real electricity prices is shown in Figure 8.
The average error pairs for each prediction model are shown in Table 3.
As can be seen from Figure 8 and Table 3, the use of DBN for electricity price prediction is more accurate than that of single-layer neural networks, which can provide an effective method for actual electricity price prediction.
5. Conclusions
In this paper, a price prediction model based on a deep belief network is proposed. Simulation predictions are made using real data from the U.S. PJM electricity market and compared with forecasting models of other neural networks. The following conclusions are drawn: the prediction accuracy of the deep belief network model used in this paper is higher, and the use of deep belief network can provide an effective method for China’s electricity sales companies to predict electricity prices.
Data Availability
This paper has no data support.
Conflicts of Interest
The authors declare that they have no conflicts of interest.