Abstract
As the development of smart grids is increasing, accurate electric load demand forecasting is becoming more important for power systems, because it plays a vital role to improve the performance of power companies in terms of less operating cost and reliable operation. Short-term load forecasting (STLF), which focuses on the prediction of few hours to one week ahead predictions and is also beneficial for unit commitment and cost-effective operation of smart power grids, is receiving increasing attention these days. Development and selection of an accurate forecast model from different artificial intelligence (AI)-based techniques and meta-heuristic algorithms for better accuracy is a challenging task. Deep Neural Network (DNN) is a group of intelligent computational algorithms which have a viable approach for modeling across multiple hidden layers and complex nonlinear relationships between variables. In this paper, a model for STLF using deep learning neural network (DNN) with feature selection is proposed. A wide range of intelligent forecast models was designed and tested based on multiple activation functions, such as hyperbolic tangent (tanh), different variants of rectifier linear unit (ReLU), and sigmoid. Among the others, DNN with leaky ReLu produced the best forecast accuracy. Regarding the precision of the methods used in this research work, certain output measures, such as absolute percentage error (MAPE), mean square error (MSE), and root mean square error (RMSE) are used. There was also a reliance on multiple parametric and variable details to determine the capability of the smart load forecasting techniques.
1. Introduction
Load forecasting strengthens utility corporations’ ability to model and anticipate power loads in order to maintain a balance between supply and demand, reduces manufacturing costs, estimates fair energy pricing, and regulates capacity scheduling and future planning. These forecasts are extremely important for energy suppliers and other power system stack holders, as well as for power generation, transmission, and distribution industries. There is also the precise projection of electric load magnitudes and geographic locations for various times of the planning horizon [1]. The main criterion is used to test the predictions of the model on the basis of lead-time horizon. Accurately predicting future load requirements is critical for proper generation planning. It additionally improves the performance of the power system and facilitates managerial decision making in the future. Inaccurate forecasts can be the reason for massive economic losses for housing and power system. Researchers have applied a number of techniques for electrical load demand forecasting using numerous statistical, mathematical, and artificial intelligence-based approaches to facilitate the supply chain of electrical load in a smooth manner. It is found that deep neural networks (DNN) and their hybrid combinations with other meta-heuristic optimization algorithms provided wonderful functionality in managing complicated nonlinear relationships, model complexity, and their prediction performance is found accurate as compared to other conventional methods [2]. The major objective of this research work is to enhance electrical load demand forecast accuracy by implementing the state-of-the-art deep neural networks using LSTM and RNN architectures. In particular, the impact of seasonal variation on forecast error has been explored and reported.
According to the literature, short-term electrical load demand forecasts are of considerable interest. This forecasting is important for power system control, unit input, security assessment, economic calculations, and power markets [3]. STLF is under high consideration for controlling and optimizing energy systems on a daily energy efficiency basis, exchange, and security checks. It is also useful for reliability considerations and mathematical calculations in the power system. However, STLF needs a great effort to produce reasonable forecast accuracy because of the lesser lead-time. For immediate and accurate future predictions on the basis of lesser lead-time, we need more parametric analysis and complex modeling techniques [4]. Choosing a good technique for STLF is most important for high accuracy in the results. One of an electric company’s key jobs is to precisely estimate load demand at all times, which is especially important in the near term. Observing the behavior of near-future load demand may be highly useful for the assessment and operation of power systems in terms of a noninterrupted supply chain of power [5].
There are several load-forecasting techniques that are classified as parametric and nonparametric techniques in two major sections. Parametric techniques are based on mathematical and statistical equations such as time series and linear regression. Nonparametric techniques are artificial intelligence and machine learning-based techniques such as artificial neural networks (ANN), deep neural networks (DNN), fuzzy logic, and expert systems. In the category of nonlinear techniques, many hybrid combinations of ANN and DNN with nature inspired meta-heuristic techniques such as genetic algorithm (GA), particle swarm optimization (PSO), feature selection, and others have been reported frequently for STLF in the past decade. It is further reported by many researchers that these hybrid combinations of intelligent forecast methods produced highly efficient models in terms of accuracy and generalization.
The electric load forecasting is categorized into three classes including short-term forecasts, that is, from few minutes to few days ahead, medium-term forecasts from one week to few months ahead, and long-term forecasts of 1 year to 10 years ahead [6]. Short-term load forecasting (STLF) is useful for day-to-day decisions including fuel requirement and maintenance scheduling systems setup, whereas medium-term load forecasting (MTLF) is important for system maintenance, purchasing electricity, and pricing plans. This maintains the shutdown and maintenance scheduling, as well as load-switching operations. On the other hand, long-term load forecasting (LTLF) is highly beneficial for expansion plans and the development of new power plants.
In this study, intelligent computational models are designed and developed using a deep neural network integrated with feature selection and genetic algorithm using various activation functions, such as sigmoid, tanh, and ReLU to forecast short-term electrical energy demand. To make forecasts more trustworthy, all significant factors impacting future power usage must be included [7]. DNNs are always difficult to train, test, and validate, particularly when the dimensions of the input are very large. It is very critical to pick important features by evaluating a DNN-trained model’s first-layer activation potential [8]. Moreover, a crucial factor in the DNN-based model for STLF is the availability of a small number of data samples for the training phase, which can cause the model to overfit. To avoid overfitting, we used 2 years of electricity load from FESCO, a company in Pakistan, to supply the electricity. In this investigation, there are one year of Australian electricity load data and other input parameters with feature selection to train the presented DNN models utilizing a single activation function. In the literature, [9], Denil et al. demonstrate that it is possible not only to forecast all the other weights but also to exclude some of the weights, providing a few weights to every element. It is shown for neurons with multiple layers, training 25% with parameters produces the same error as learning all weights. Sainath et al. [10] use low-rank matrix factorization to reduce the number of input parameters in the final layer of a DNN.
Hybrid load demand forecast model by integrating deep recurrent neural networks and LSTM architectures are designed, developed, and tested. Their performance is compared with the conventional ANN design. The performance of the developed hybrid model in different seasonal and load demand variations is examined on a day-ahead and week-ahead basis. The integration of various meta-heuristic techniques adds up the individual features of those methods to produce the summed up benefits. However, the hybridization of multiple methods leads to complexity and affects the transparency of these models. As the LSTM model keeps a track of the vibrant recent memory states, its strength in remembering the recent past states is considered superior as compared to other meta-heuristic methods.
The remainder of the paper is ordered as, Section 2 explains DNN’s and RNNs importance for load forecasting. Section 3 explains the data description. Section 4 explains the methodology behind the hybridization of DNN and the feature selection-based model for STLF. The results and corresponding consequences are seen in Section 5, and finally, the conclusions regarding the proposed method are provided in Section 6.
2. Deep Neural Networks and Applications
Deep neural network is an advanced form of conventional ANN whose learning is typically carried out using the framework of complex architecture with multiple hidden layers and neurons. The word “deep” refers to the topological structure of NN with a number of layers in the network. A deep neural network (DNN) is just an ANN with some extra layers than the three standard layers of multiple-layer perceptron (MLP). A deep neural network integrates several nonlinear layers of computation, utilizing basic parallel operating components biologically inspired nervous systems. Deep learning is traditionally focused on using back propagation including gradient descent and a huge number of neurons and hidden layers [11]. The deep structure enhances the potential of neural networks for abstraction. Currently, the advancement of the Internet of things (IoT) and big data allows the deployment of DNNs in a variety of ways. Moreover, recent findings for DNN have shown great promise in other fields, such as computer vision and voice recognition. However, there is much less work on applying DNN to short-term load forecasting in STLF is found in the literature [12]. DNNs allow higher precision to be achieved by detecting dominant factors that influence electricity consumption trends and can surely make a major contribution to next-generation energy systems and the recently launched Smart grid [13]. A typical DNN interconnection structure with one input and output layer and multiple hidden layers having neurons is shown in Figure 1. It can be seen that there are a number of hidden layers which convert this neural network to deep neural network. The results will also be more accurate by more layers.
2.1. DNN with Genetic Algorithm
Genetic algorithm is a method of programming which derives its foundation from biological evolution [14]. The Genetic Algorithm is generally used as a problem-solving technique to have the optimized value [15]. A hybrid model designed by integrating a genetic algorithm (GA) and deep neural network (DNN) is used to increase performance, cogency, and reduce the error [16]. Specifically, GA is used for selecting features and optimizing DNN design parameters [17]. A set of possible solutions is provided to GA as inputs and evaluation of the performance of each input is carried out with a metric called a fitness function, which allows each candidate to be quantitatively evaluated. The input to the GA is a series of feasible solutions to the problem, stored in some form, and a metric named as fitness function that allows every applicant to be concretely evaluated. The GA’s functionality was proven by the creation of a DNN with more than four million parameters; the best infrastructure ever developed by an evolutionary algorithm [18].
2.2. DNN with Feature Selection
Selection of features distinguishes the relevant features from a collection of data and eliminates unrelated or less-significant features which do not lead most to our target variable in order to obtain optimal reliability for our model. It is commonly accepted that the performance of DNNs is because the relationship between the target value and the features of the input is very significant. It takes gradual and definite transformation to render useful features [19]. For a DNN, the measurement of sensitivity does not work far beyond one or two layers. Therefore, in order to better evaluate an input feature’s contribution, we review its activation potential (averaged over all input training values and hidden neurons) relative to the full activation potential. The greater the possible activation involvement of an input factor, the more likely its inclusion in the hidden layers [20].
2.3. Hybrid LSTM and DNN Recurrent Neural Network
The ANN is referred to as recurrent neural networks when feed forward neural networks are expanded to provide feedback connections as shown in Figure 2. The input neurons are responsible to receive inputs, whereas relational ends receive the signals modified with an activation function from the current input layer and from the hidden nodes in the previous state of the network at each time-step of sending input through a recurrent network. Long short-term memory (LSTM) networks are a revamped variant of recurrent neural networks, allowing memory retrieval of previous data simpler. The RNN problem of the vanishing gradient is solved here. Given unpredictable time delays, LSTM is well suited for categorizing, analyzing, and forecasting time series. It trains the model using back-propagation. The performance is determined by the secret state of the hidden layers. The concept behind RNNs is to make use of the knowledge in sequence. It is generally assumed that in a typical neural network all inputs and outputs are distinct of one another.
3. Research Methodology
The main requirement for an accurate prediction model is careful analysis of the load data and its dynamics. A big quantity of data is being gathered with the aid of the intelligent meters on every day basis which is called raw data at initial stage as shown in Figure 3. Big statistics analytics can be helpful in reaching insights for smart grid energy management [4]. To achieve the good forecast results, variation and the behavior of the load data is of high consideration. Initial steps for treatment of data are data preprocessing which is also called the data normalization. These methods can be carried out according to simple load profile analysis. On the basis of traits of input data, it can be classified into distinctive clusters by which the network performance can be increased. To preprocess data, the first stage is the compilation of data from the different information systems, for instance, the equipment, customer and charging details, weather, and electrical load system [21]. As designing data management and smart electronics increases, data-oriented applications have been gaining far more interest in both academia and industry through power converters in power grid companies [22, 23]. In addition to the load demand data other factor are also important including metrological data and day type information.
Long short-term memory (LSTM) networks are an improved variation of recurrent neural networks that make it easier to retrieve earlier data from memory. The declining gradient issue is handled here. Given the unpredictable nature of time delays, LSTM is ideal for categorizing, analyzing, and predicting time series. The performance of the LSTM-based intelligent forecast models has been proved in many smart systems such as smart grids.
3.1. Effect of Temperature
It is noticed that the electrical load demand increases with the rise in temperature during the summer season and it decreases in the cold season. Therefore, the seasonal variables should be included in the predicted model input to obtain accurate predictive results. A review of the literature shows that there is a strong correlation between seasonal variables and load demand. The results of human sensitivity test tell that dew point between 40 F and 60 F is considered comfortable for the humans [24]. The demand for load remains normal in this range of dew point. There is a more need of strength when the temperature falls below 10°C due to heating requirements in a family.
3.2. Working and Nonworking Days
Electricity usage is higher on weekdays while electricity consumption is low on Saturday and Sunday, and also on other public holidays. The “Working Day feature” is chosen based on these results to draw this impact.
3.3. Impact of Time
There is high impact of time on electricity usage. Energy usage values reflect an up-and-down trend, respectively, during both middays. To express the time dependence as hour and day of week, two functions are extracted.
3.4. External Factors
The external factors can also influence the power load behavior (we define data collected as external factors outside the energy database) such as season, climate, and holiday statistics [25].
3.5. Data Preprocessing
There is a process called preprocessing by which the input data is converted into normalized form to facilitate the NN for easy interpretation of input patterns for better results. The change between each input data point interval is between 0 and 1 throughout the normalization procedure. Each input’s data can be transformed into normalized form independently or in groups. Preliminary work on entering the input data reduces the size of input space to DNN, which lowers the training time of the network. It shortens the input surface measurement and reduces the variety of parameters that need to be set for the training process.
3.6. Training and Test of Datasets
The model is trained before testing to forecast the input data at high accuracy. We also separated the input data into training and testing datasets in this model, utilizing two years of data for training and one year of data for testing. From the dataset testing, we used 24 and 168 hours forward records for day and week ahead prediction, respectively.
3.7. Training and Test of Datasets
In the design and development of hybrid forecasting models based on DNN and multiple meta-heuristic techniques, different activation functions were used which directly affect the behavior and ultimate DNN efficiency. DNNs typically need capacities for nonlinear activation. Because of their simplicity, rectified linear units (ReLU) are commonly used in modern-day DNNs. Arunadevi et. al. [26] researched the impact of activation function on classification accuracy using DNN. However, choosing an appropriate activation feature is a difficult task [27–29]. Some of the commonly employed activation functions are given in the subsequent sections.
3.8. Sigmoid Function
A sigmoid function is a type of activation function, more precisely a squashing function. Crushing functions, as seen in Figure 4, limit the output to a range of 0 to 1, making them effective in probability prediction.
3.9. Linear Rectified Unit (ReLU)
The rectified linear activation function or ReLU for short is a linear piece-by-piece function that directly outputs the input if it is positive, otherwise, it will output zero as shown in Figure 5. For several forms of neural networks, it has become the default activation function, since a model that uses it is easier to train and often achieves better performance. We have, f(x) = m(o, x).
3.10. Leaky ReLU
Leaky ReLUs are such method to overcome the “dying ReLU” problem. Rather than making the feature zero if x < 0. Instead, a leaky ReLU would have a slight negative slope (of 0.01, or so), as shown in Figure 6. It can be expressed mathematically:
F(x) = 1 (x < 0) (αx) + 1 (x ≥ 0) (x).
Here the α is a constant of computation.
4. Results and Discussion
This section presents the electrical load demand prediction results of benchmark combinational approaches using conventional ANN, RNN, and LSTM. The simulation results, as well as the pertinent discussion of the suggested prediction model for various forecast scenarios, are also presented. To guarantee that the model works successfully in different seasonal fluctuations, its forecast accuracy is validated using load demand and meteorological data for all four seasons of the year. Furthermore, to ensure that the model does not overfit, the forecast performance is validated under high variable load demand situations one day and one week ahead, as well as seasonal load changes.
The model's performance in the aforementioned varied situations demonstrates that it is capable of providing strong prediction results under the vivid and vibrant settings of load demand. To explore the influence of these tactics, a relative analysis of the aforesaid methodologies is performed with respect to the appropriateness of the input variables and ANN design optimization. On a one-hour sampling frequency, the data are collected at a rate of 24 samples per day and 168 samples per week, and it contains electrical load as well as four meteorological variables: dry bulb temperature, wet bulb temperature, dew point, and humidity. MSE and MAPE are performance measures employed to analyze and compare the efficacy of various methods.
The variations in load demand are analyzed w.r.t. seasons: the load demand in the spring and fall seasons is lesser than the load demand in the winter and summer seasons. Furthermore, the summer season's load is less consistent, having higher peaks than the winter season's load. This disparity is most likely due to the less frequent usage of air conditioning during extremely hot summer days, as opposed to the more consistent use of heaters throughout the winter. Based on this trend, the data have been divided into four seasons: summer is seen as lasting from November to January, autumn from February to April, winter from May to July and spring from August to October. In addition to the performance indicators, the number of repetitions for the same training error are also used to evaluate the prediction accuracy of the presented algorithms.
4.1. Prediction by ANN Model
A three-layer neural network with an 8-16-1 topology will be used in the tests. The transfer function for hidden layer neurons is logistic sigmoid; however, it is linear for output layer neurons. Figure 6 shows the projected load demand assessment of the ANN model for twenty-four hours ahead load during the summer. The graph’s X-axis indicates hourly time with an interval of one hour, while the actual and predicted load demand can be seen on Y-axis. It is apparent that load demand fluctuates depending on the time of day, starting from modest in the morning; however, it rises as the day activities are started. The results of summer season forecasting of one-day ahead of ANN model for other seasons including winter, autumn, and spring are shown in Figures 8–10, respectively. In this model, the best results are found in winter season, where MSE remained 0.10015 and MAPE is found 1.31% for day-ahead predictions.
4.2. Prediction by LSTM Model
The results of LSTM-based model are presented in Figures 11–14 for the autumn, spring, summer, and winter seasons, respectively. The red line shows the actual load and the green line shows the predicted values. The minimum forecast results for this model are observed in the summer season, where MSE is observed as 0.09153 and MAPE is 1.02% for 24 points per day. The predicted load is decreasing at the initial, but there is a difference between both lines giving better prediction results.
4.3. Prediction by RNN Model
The results of RNN-based hybrid model are presented in Figures 15–18 for the summer, winter, autumn, and spring seasons, respectively. The minimum forecast errors for this model are observed in the summer season, where MSE is observed as 0.09873 and MAPE is 1.23% for 24 points per day. The predicted load is decreasing at the initial, but there is a difference between both lines giving better prediction results. Figures 15–18 show the forecast error graphs of the RNN-based hybrid model for summer, winter, autumn, and spring seasons, respectively. The load pattern of all four seasons is different because of the changes in the meteorological parameters, such as temperature, humidity, and cloud cover. However, the proposed model shows reasonable forecast accuracy for all the seasons and demonstrates its generalized prediction strength throughout the year under different load demand conditions. The load forecast results in terms of MAPE are summarized in Table 1.
Among the three models deployed for the electrical load demand prediction, it is observed that the hybrid models based on the combination of LSTM and ANN and RNN performed better w.r.t. forecast accuracy for one-day ahead forecasts. Especially, the RNN model predicted the electrical load demand with an accuracy of 1.01% in terms of MAPE. The RNN model is applied for the prediction of one week ahead forecast, and the results for the summer season are depicted in Figure 19. The model predicted the load demand with a reasonable forecast accuracy of 1.09% MAPE on weekahead bases. All these results show the superiority of the hybrid models in terms of forecast accuracy and generalization.
5. Conclusion
There are several feature descriptors currently available that provide high-dimensional features to identify the behavior in the video, but it takes detailed research to measure the impact of those features on classification. Although size reduction techniques are available to reduce the dimensions of items, their main focus is good reconstruction and the prejudicial information is lost in low-dimensional space. We have used the three types of modeling as LSTM modeling, RNN modeling, and NN modeling for one day forecasting of all the seasons. There is more accuracy using the leaky ReLU activation function with RNN. There are also good results for yearly forecasting data by using the above techniques. The training data is undertaking preprocessing step to predict the new features that will be more important for the use of electricity. The proposed hybrid forecast models have shown high forecast accuracy and generalization that would lead to less-operating costs and safe operation of the power utility companies.
Data Availability
No data are used in the study.
Disclosure
The statements made and views expressed are solely the responsibility of the authors.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Authors’ Contributions
All authors equally contributed in this article.