Abstract
In this paper, a novel dynamic programming technique is presented for optimal operation of a typical renewable microgrid including battery energy storage. The main idea is to use the scenarios analysis technique to proceed the uncertainties related to the available output power of wind and photovoltaic units and dynamic programming technique to obtain the optimal control strategy for a renewable microgrid system in a finite time period. First, to properly model the system, a mathematical model including power losses of the renewable microgrid is established, where the uncertainties due to the fluctuating generation from renewable energy sources are considered. Next, considering the dynamic power constraints of the battery, a new performance index function is established, where the Lagrange multipliers and interior point method will be presented for the equality and inequality operation constraints. Then, a feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution. Finally, simulation and comparison results are given to illustrate the performance of the presented method.
1. Introduction
Nowadays, renewable energy sources (RESs) such as wind or photovoltaics have become more wide spread due to needs for satisfying the environment concerns. On the other hand, distributed generators (DGs) like diesel engines, microturbines, and fuel cells can be used to enhance the resiliency of power system and yield other social economic benefits. Therefore, renewable microgrid is expected to play an important role in future power systems [1, 2]. As a key enabling element of renewable microgrids, battery energy storages make microgrid become a strong coupling system in the time domain. In this regard, the methodologies applied to operation management of a renewable microgrid are getting more complicated and challengeable; therefore there is a strong need for more reliable scheduling of energy sources in renewable microgrid including battery energy storage.
So far, many researchers have dealt with the optimal operation scheduling of energy sources in microgrids [3–7]. Previously conventional mathematical programming such as Lagrange relaxation [8, 9], lambda iteration [10, 11], Newton-Raphson [12], interior point method [13], weighted minimax [14], and quadratic programming [15] have been used to determine the least cost solution. However, the conventional mathematical programming methods have major disadvantages such that they can be trapped in local optimal, exhibited sensitivity to the initial starting points. And many of the methods cannot solve the nonsmooth, convex, and nonmonotonically increasing cost functions. Recently, computational intelligence [16, 17] and artificial intelligence based nonconventional methods [18] have been used to solve the optimal operation scheduling of energy sources in microgrids. Artificial intelligence based methods such as artificial neural network and computational intelligence methods such as genetic algorithm, particle swarm optimization, harmony search, simulated annealing, differential evolution, gravitational search algorithm, biogeography based optimization, bacterial foraging algorithm, ant colony optimization, cuckoo search, bat algorithm, artificial bee colony, firefly algorithm, and flower pollination algorithm have been used to solve the problem. These methods can enable us to solve the nonlinear and no-convex cost functions and can obtain nearly the global solutions. However, these methods have major disadvantages such as the evolutionary algorithms greatly depending on their parameters and having high computational time. Besides, hybrid methods which combine different algorithms have been used to solve the optimal operation scheduling of energy sources in microgrid. But, these methods usually have long computational time. Also, all previous researchers search the optimal solutions in each separated time periods. In other words, they are static optimal operation scheduling which do not consider the relationships among different time periods. Dynamic optimal operation scheduling problem is another fundamental part of the renewable microgrid operation to maintain the power balance [19–24]. Among these methods, Cheng et al. [23] have used an enhanced quorum sensing based particle swarm optimization to deal with the dynamic operation optimization problem. However, it is hard to solve the complex optimization problem with high dimension variables and multiple constraints by such metaheuristic methods. Liu et al. [19, 22] have used the dynamic programming algorithm for handing the dynamic economic dispatch problem. Generally, dynamic programming can be used to solve nonlinear dynamics when the problems can be discredited with time, state, and decision variable [25]. In other words, dynamic programming can be used to solve the problem by dividing the decision process into different stages. Meanwhile, these stages interact and interconnect with decision variables. And the target of the dynamic programming is used to find the decision set over the whole searching trajectory. However, such methods may have some deficiencies handling large scale problems. And a challenging aspect of this method is determining which of the inequality constraints are binding at the solution. Another relevant aspect in renewable microgrid operation management should be coping with uncertainty in the renewable energy sources, load demand, and market prices [24–26]. Similarly, power line losses should also be considered in the renewable microgrid system. There have been some researches including the power losses in a renewable microgrid [5, 27, 28]. For example, the output power losses of distributed generators are modeled in articles [5, 29]. The power line losses which are calculated using B-coefficients are considered in the economic dispatch problems [30, 31].
This paper presents several contributions as follows:
The power losses of the renewable microgrid are included. It is noteworthy that, from an optimal operation planning point of view, such method can be optimized a renewable microgrid globally.
Uncertainty in the renewable energy resources is considered. A methodology based on scenario analysis is used to proceed the uncertainties related to the available output power of wind and photovoltaic units.
The dynamic performance of the battery storage system is considered. Therefore, this dynamic operation constraint makes renewable microgrid to become a strong coupling system in the time domain. Then, the Lagrange multipliers and interior point method will be presented for these operation constraints.
A feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution. The advantage of the scheme is that the number of numerical operations is linear in the number of optimization parameters, which enables one to solve for a smooth input/control history. Meantime, this scheme enables one to initialize the optimization problem with a zero-initial guess. Finally, this scheme is a direct method to obtain the same results.
2. Description of the Renewable Microgrid
2.1. Renewable Microgrid
In this paper, the renewable microgrid system considers diesel generators, microturbines, fuel cells, wind turbines, photovoltaics, and a battery energy storage system in Figure 1. The renewable microgrid can operate in parallel to the main grid or as an island.
2.2. Active Power Balance of the Renewable Microgrid
In general, the active power balance between electrical energy production and consumption must be met at each time interval expressed as follows [28, 33]:where , are the power set-point for the wind turbines, photovoltaics, diesel generators, microturbines, fuel cells, battery energy storage, power exchange between the main grid and renewable microgrid, controllable distribution generators, and load levels. is the power loss of the renewable microgrid. are the total number of wind turbines, photovoltaics, diesel generators, microturbines, fuel cells, battery energy storage, and load levels. In this paper, the controllable distribution generators (DGs) included diesel generators, microturbines, and fuel cells.
2.2.1. Estimation of Power Losses Coefficients
When the power losses of the renewable microgrid are considered, these power losses coefficients can be calculated as follows [28, 34]:where and and are the power losses coefficients.
2.3. Subsystem Model of the Renewable Microgrid
This paper considers a renewable microgrid containing several types of devices, such as battery energy storages, distributed generators, wind turbines, and photovoltaics. A block diagram is given in Figure 1.
2.3.1. The Model of Battery Energy Storage System
Battery energy storage can store energy when charging. On the other hand, it can supply energy to the load demand when discharging.
The degradation cost of battery energy storage can be calculated as follows [33]:where is the cost coefficient which can be determined by battery energy storage charging or discharging power.
is the permitted rate of charge during a definite period of time and is the permitted rate of discharge. The following equation can be expressed for a battery energy storagewhere or indicate the working mode of the battery energy storage, . Define or if the battery energy storage is charge or discharge. These variables satisfy
The permitted rate of charge and discharge are limited by the following constrains:where and are the maximum charging and discharging rates of battery energy storage.
After charge or discharge, the state of charge of the battery energy storage can be calculated as follows [33]:where are the energy conversion coefficients. is the energy capacity of the battery energy storage. On the other hand, the state of charge of the battery energy storage must be maintained in the following range. is the operating time of the battery energy storage.where are required limits of the state of charge.
2.3.2. The Model of Controllable Distributed Generators
In this paper, the distribution generators (DGs) are included diesel generators, microturbines, and fuel cells.
(A) Diesel Generators. In general, diesel generators serve as a backup power source. Meanwhile, the operating cost of diesel generators is formulated as follows:where is the fuel cost of diesel generator; is the operating and maintenance cost; is the start-up cost of diesel generator.
The fuel cost of diesel generators is modeled as a function of their actual output powerwhere is the actual output power. are the fuel consumption curve fitting coefficients.
The operating and maintenance cost can be expressed aswhere is the operating and maintenance cost coefficient. is the operating time of diesel generator.
The start-up cost of diesel generator is relative to its operating statewhere are the hot start-up cost and cold start-up cost. is the cooling time of diesel generator. are the off time and shut down time of diesel generator.
For a stable operation, the actual output power of diesel generator is limited by lower and upper bounds as follows:where is the minimum active power of diesel generator. is the maximum power of diesel generator.
The following constraint ensures diesel generators do not exceed their ramp limits:where is the ramp constraint coefficient.
Due to the physical constraints that state shifting can only take place after a fixed time interval, the state variable of two adjacent times is as follows:where are the cumulative uptime and the minimum turn on time.
(B) Fuel Cells. Fuel cell directly converts chemical energy to electrical energy by electrochemical reactions, which is one of the most promising energy conversion technologies. The operating cost of fuel cells are formulated aswhere is the fuel cost of fuel cell; is the operating and maintenance cost; is the start-up cost of fuel cell.
The fuel cost of fuel cells is modeled aswhere is the fuel price; is the operating time of changing two adjacent states; is the consumption with active power for the auxiliary equipment. is the efficiency of fuel cells.
The operating and maintenance cost can be obtained bywhere is the operating and maintenance cost coefficient.
The start-up cost of fuel cells can be expressed aswhere is the hot start-up cost of fuel cells; is the cold start-up cost of fuel cells. is the off time of fuel cells; is the cooling time of fuel cells.
The actual output power of fuel cell is limited by lower and upper bounds aswhere are the minimum and maximum active power of fuel cells.
The following constraint ensures fuel cell does not exceed their ramp limitswhere are the minimum and maximum ramp rate of fuel cells.
The minimum on or minimum down time constraints for fuel cells can be expressed aswhere is the cumulative uptime. are the minimum turn on time and shut down time of fuel cells.
The number of turn on and turn off can be expressed aswhere is the number of turn on and turn off in time ; is the maximum number of turn on and turn off.
(C) Microturbine. Microturbines have the higher power density, produce less noise, and emit much less pollutants, especially . So microturbines are effective devices of converting the fuel energy into electrical energy. Meanwhile, the operating cost of microturbines are formulated as follows:where is the fuel cost of microturbine; is the operating and maintenance cost; is the start-up cost of microturbine.
The fuel cost of microturbines is modeled aswhere is the fuel price of microturbines; is the efficiency of microturbines.
The operating and maintenance cost of microturbines can be expressed aswhere is the operating and maintenance cost coefficient of microturbines.
The start-up cost of microturbines can be expressed aswhere is the hot start-up cost of microturbines; is the cold start-up cost of microturbines. is the off time of microturbines; is the cooling time of microturbines.
The actual output power of microturbines is limited by lower and upper bounds as follows:where are the minimum and maximum active power of microturbines.
The following constraint ensures microturbines do not exceed their ramp limitswhere are the minimum and maximum ramp rate of microturbines.
The minimum on or minimum down time constraints for microturbines can be expressed aswhere is the cumulative uptime; are the minimum turn on time and shut down time of microturbines.
2.3.3. The Probability Model of Wind Turbines
Wind powers as renewable energy sources (RESs) are dependent on numerous factors such as wind velocity and efficiency. The approximate relationship between the wind power and wind speed can be expressed aswhere are the cut-in wind speed, cut-out wind speed, and rated wind speed. is the rated output power of wind turbines.
Prior research has shown that the wind speed profile follows the Weibull distribution over time, which is [29, 32]where positive variables and are the scale parameter and shape parameter, respectively.
Based on the characteristic of the wind power (34) and the probability distribution function (35), the probability distribution function of wind power is obtained by
2.3.4. The Probability Model of Photovoltaics
The output power of photovoltaic can be calculated by its rated output power at the standard test condition and the operating ambient temperature [35]where is the array surface area; is the efficiency of the photovoltaic in realistic reporting conditions; is the irradiance on a surface with inclination to the horizontal plane; are the parameters that depend on inclination , reflectance of the ground , etc.where is the ratio of beam radiation on the tilted surface to that on a horizontal surface at any time and is the solar radiation, for that day, both referring to a horizontal surface; is ratio, diffuse radiation in hour or diffuse in day; are the parameters of the daily clearness index .
Many researches have proved that cloudiness is the main factor affecting the difference between the values of solar radiation measured outside the atmosphere and on earthly surface. The daily clearness index can be obtained by [36]where are the ratio of the irradiance on horizontal plane and the extraterrestrial total solar irradiance.
The effect of clouds on terrestrial irradiance is the daily clearness index; can not be predicted with complete confidence; it must be treated as random variablewhere are the lower and upper bounds of the observed range for ; is the parameter of daily clearness index.where are the parameters of daily clearness index.
In particular, if , the probability density function can be expressed aswhile if , the probability density function can be obtained bywhere .
2.3.5. The Model of Interaction with External Main Grid
In this paper, the renewable microgrid is connected to the external main grid and can trade energy with the main grid. The transaction incurs the following cost to the renewable microgrid:
Let the amount of energy be bounded bywhere is the maximum transaction limits; are the price coefficients to purchase and sell energy.
2.4. Dynamic Operation Model of the Renewable Microgrid
Dynamic economic operation management of the renewable microgrid is to determine output power of distribution generators, in order to minimize total operation cost of the renewable microgrid and meet the dynamic operation constrains.
The total operating cost of the renewable microgrid can be defined aswhere is the decision or control variables (specifically, action at state in period ); is the state variables of the renewable microgrid; is the random variable such as the output power of wind turbine or photovoltaics.
can be defined as follows:
The state variables of renewable microgrid in period can be defined as follows:
The operation constraints of the renewable microgrid can be expressed as power balance (1)-(2), battery energy storage limits (5)-(8), distributed generations limits (13)-(16), (21)-(25), (30)-(33), and interaction with external main grid limits (50).
2.4.1. The Standard Formulation of the Dynamic Operation Model
According to dynamic programming formulation, the dynamic operation model of the renewable microgrid can be formulated as
The operation constraints can be expressed aswhere are, respectively, the equality and inequality constraints.
Meanwhile, the constraint of condition should be satisfied as follows:
3. Solution Methodology
Based on Section 2, dynamic operation management of the renewable microgrid can be regarded as a discrete time system under uncertainty. In order to solve the proposed problem, the random variables can be realized by the scenario analysis technique. Then, a feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution. The flowchart of the whole process is given in Figure 2.
3.1. The Scenario Analysis Technique
In this paper, a discrete set of scenarios can be used to represent the probability realization of the output power of wind turbines or photovoltaics. On the other hand, these scenarios are generated using the Roulette wheel mechanism and Monte Carlo simulation method [19]. And the probability distribution function of the random variables can be obtained by (36) and (47)-(48).
The Lattice Monte Carlo simulation(LMCS) can be used to generate the random numbers [29]where is the number of random variables; is the number of random sampling; is vector with dimension .
According to the desired preciseness, the probability distribution functions (36) and (47)-(48) are divided into class intervals. Each class interval determines mean value ; and each interval is associated with a probability denoted by . Meanwhile, the probabilities of different intervals are normalized in which their summations become equal to unity.
Therefore, each scenario comprises a vector identifying the output power of wind turbine or photovoltaics:where are binary parameters indicating where wind power interval or photovoltaic power output whether are selected in scenario. On the other hand, comparing the random number which follows the LMCS strategy and the probability , the binary parameter can be selected.
Thus, the output power of wind turbine or photovoltaics for each scenario can be obtained by
The normalized probability of each scenario can be expressed as follows:where is the number of scenarios.
3.2. The Feedback Control Scheme Based on the Dynamic Programming
In the following section, we formulate the dynamic operation management of renewable microgrid which satisfies dynamic operations constraints. Firstly, the Lagrange multipliers and interior point method will be presented for the equality and inequality operation constraint. Then, a feedback control scheme based on the dynamic programming is proposed to solve the model and obtain the optimal solution.
3.2.1. The Sequential Quadratic Programming Subproblem Formulation
In this paper, the sequential quadratic programming will be presented for the formulation (52)-(61). The purpose of sequential quadratic programming is to leave the model with only linear and quadratic terms. The equivalent formulation can be obtained as follows:where is a constant. are the perturbation variables. are the nominal values of . And these variables can be introduced and defined as
The operation constraints can be expressed aswhere and can be obtained in Appendix A.
3.2.2. Interior Point Methods
According to the sequential quadratic programming formulations (67), by applying the Lagrange multipliers and slacking variables to proceed the equalities and inequalities operation constraints, a new formulation is shown aswhere is considered as the kth iteration. is a constant.
Meanwhile, the parameters can be defined as follows:The approach in solving is given in Appendix B.
3.2.3. Total Algorithm Procedures
In this paper, an algorithm is presented based on an adaptation of Mehrotra’s predictor-corrector algorithm [37]. Given the initial parameters and the residual terms are defined as
As the solution set approaches optimum point, the summation of all residual term should approach zero.
The total algorithm can be expressed as follows.
Step 1. Calculate the residual term aswhere the expressions for are given in (78)-(81). If , then go to Step 5.
Step 2. Calculate the affine-scaling direction by using the method introduced in Appendix B.
can be defined as
The scalars , and can be calculated as follows:where is the dimension of .
Meanwhile, the centering parameter is defined as follows:
Step 3. Calculate the combined predictor-centering-corrector direction and the scalar as the method introduced in Appendix B.
Letwhere .
Calculate the scalar as follows:
Step 4. Calculate from the equationwhere . Then set and return to Step 1.
Step 5. Calculate the search direction and the optimal solutionCalculate all state variables by using the equation
4. Simulation
In this section, the renewable microgrid in the simulation is shown in Figure 1; it operates in parallel to the main power grid or as an island and comprises wind turbines with maximum power of 13kw, PV panels with maximum power of 15kw, and distributed generators such as microturbines, diesel generators, and full cells. A battery energy storage is included, bounded between 20 and 60 kWh and with maximal charge and discharge rates, respectively, 30 and -30kw. The charge and discharging efficiencies are both equal to 0.85. Tables 1–4 describe the distributed generators units parameters, based on data provided in [29, 32]. The simulations are carried out in the MATLAB environment on an Intel Core 2 Duo 3.00GHz running Windows 7. In the simulations study, we chose the sampling time of 1 hour. And simulations are performed over a horizon of 24 hours.
The electricity usage, wind power, and solar generation data in the simulations have been provided by National Renewable Energy Laboratory. The electricity usage data is from the utility operator customers with peak usage over 90kw; the maximum power demand of load is 90.1kw. The daily electricity usage (from Miami, FI, on a certain day) is shown in Figure 3.
4.1. Scenario Generation and Reduction Results
The approach used for generation of scenarios may have an impact on the determination of operational cost of renewable microgrid. In the study case, the dates from wind power profiles of the 24 hours are used as scenarios in the dynamic operation management of renewable microgrid. The scenarios of wind power and photostatic chosen for this study are shown in Figures 4, 5, and 6.
4.2. The Base Case of the Dynamic Operation Management of Renewable Microgrid
In this part, its assumed that the renewable microgrid is disconnected from the main power grid. The active power of load is supplied by the microgenerators such as microturbines, diesel generators, fuel cells, wind turbines and photovoltaics, and a battery storage system. The optimal results of the dynamic operation management of renewable microgrid are given by Figures 7 and 8.
Figures 7 and 8 show that microturbine and fuel cell have to generate more electricity power than the diesel generators. Because the diesel generators are expensive, they are restricted to their minimum value during most hours.
4.3. The Case of the Renewable Microgrid Operates in Parallel to the Main Power Grid
In the second case, its assumed that the renewable microgrid can operate in parallel to the main power grid. When the renewable microgrid is connected to the main power grid, its loads receive power from both the main power gird and the local distributed energy resources such as solar and wind power, distributed energy generators, and a battery storage system. The optimal results of the dynamic operation management of renewable microgrid are shown in Figures 9 and 10.
It should be noted that, in Figures 9 and 10, the main power grid has to generate more electricity power. Fuel cell and diesel generators are more expensive.
In Table 5, it is worthwhile to note that the total operation cost of second case is the lowest. In other word, the renewable microgrid which operates in parallel to the main power grid decreases the operation cost because of its loads receiving power from both the main power gird and the local distributed energy resources.
4.4. The Case of Dynamic Operation Management of Renewable Microgrid under Uncertainties
In this case, a methodology based on scenarios analysis is used to proceed the uncertainty of the power output of distributed renewable generators, such as wind power and photovoltaics power and it is assumed that the renewable microgrid is disconnected from the main grid. The optimal results of the dynamic economic operation optimization of renewable microgrid are shown in Figures 11 and 12.
In Table 6, it is worthwhile to note that this scheme is a direct method to obtain the optimal solution without probability. And the total operation cost of the method of proposed by this paper is the lowest. In other words, modeling the renewable microgrid system under uncertainty decreases the operation cost. This is because of considering different scenarios in the stochastic model instead of single scenario in the deterministic scheme.
5. Conclusions
In this paper, we introduce a novel dynamic operation management of renewable microgrid. First, to properly model the system, a mathematical model including power losses of the renewable microgrid is established. Then, the uncertainty in the distributed renewable generators, such as wind power and photovoltaics power, are modeled by choosing a set of possible scenarios. And each scenario is assigned a probability that reflects the scenario to be occurred. Finally, the problem of economic operation of renewable microgrid is solved by a feedback control scheme based on the dynamic programming. To solved the deterministic model efficiently, we use the Lagrange multipliers and the interior point method to solve the equality and inequality operation constraints. The numerical results indicate the validity of this optimization methodology.
Appendix
A. A Dynamic Programming Method for Constrained Optimal Control
A.1. Problem Formulation
In this paper, the discrete time, optimal control problem is presented as formulation (52)-(61). Find the controls and the initial state which minimize the function
The constraints are as follows:where the functions are assumed as twice differentiable.
A.2. Sequential Quadratic Programming Subproblem Formulations
The discrete time, optimal control problem can be approximated by a series of quadratic subproblems. And consider the Lagrange function associated with ((A.1)–(A.4)) and expressed as follows:where are the Lagrange multipliers.
Expanding (A.2) and (A.5) as Taylor series about yields the following quadratic approximation of Lagrange function:And the constraints are as follows:where and can be obtained by
are obtained from the recursive equationwhere refers to the element in row and column of the matrix . Similarly, refers to the element in row of the vector .
B. A Modification Dynamic Programming for Equivalent Problem
Expanding (A.2) and (A.5) as Taylor series about yields the following quadratic approximation of Lagrange function:andwhere is a constant and
The matrix can be defined as follows:where is a permutation matrix. For purposes of numerical stability, the number of columns in is set equal to the number of diagonal elements in which are less than unity.
So, the parameters of (75) can be expressed as follows:
And then, a modification of the dynamic programming algorithm is presented to solve the equivalent problem (75). We can obtainwhere are calculated from recursive equations.
The matrix is factored as follows:where is an orthogonal matrix of eigenvectors.
For convenience, the eigenvalues can be obtained by
Define
is a positive scalar.where .
The modification of the dynamic programming algorithm to solve the problem can be obtained by the following.
Step 1. Calculate the parameters as follows:where the parameters can be obtained asfor , storing the terms .
Step 3. Solve the linear equation as follows:
Step 4. Calculateand set .
This algorithm is used to determine the controls , initial state , and the Lagrange multipliers . Equations (B.3) and (B.15) are then used to calculate the states and the Lagrange multipliers for . The slack variable is given by
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by Key Program of National Natural Science Foundation of China (61164015, 61305132, and 61703231).