Abstract
This survey provides a comprehensive analysis on recent research related to optimization and simulation in the new paradigm of power systems, which embraces the so-called smart grid. We start by providing an overview of the recent research related to smart grid optimization. From the variety of challenges that arise in a smart grid context, we analyze with a significance importance the energy resource management problem since it is seen as one of the most complex and challenging in recent research. The survey also provides a discussion on the application of computational intelligence, with a strong emphasis on evolutionary computation techniques, to solve complex problems where traditional approaches usually fail. The last part of this survey is devoted to research on large-scale simulation towards applications in electricity markets and smart grids. The survey concludes that the study of the integration of distributed renewable generation, demand response, electric vehicles, or even aggregators in the electricity market is still very poor. Besides, adequate models and tools to address uncertainty in energy scheduling solutions are crucial to deal with new resources such as electric vehicles or renewable generation. Computational intelligence can provide a significant advantage over traditional tools to address these complex problems. In addition, supercomputers or parallelism opens a window to refine the application of these new techniques. However, such technologies and approaches still need to mature to be the preferred choice in the power systems field. In summary, this survey provides a full perspective on the evolution and complexity of power systems as well as advanced computational tools, such as computational intelligence and simulation, while motivating new research avenues to cover gaps that need to be addressed in the coming years.
1. Introduction
Energy is essential to assure most of the activities in developed societies [1]. Looking to the energy vectors, electricity consumption is rising significantly and changing the global paradigm of the energy mix [2]. The emerging and developed countries are also changing the primary sources to obtain the electricity, supporting the increase of the use of renewable primary sources, such as wind and the sun and the use of more clean sources like the natural gas [3]. The growing responsibility of each country regarding environmental aspects is also contributing to this paradigm change [4].
Other important aspects of modern societies are the efficiency in the use of energy, and electricity in particular, and the transparency in electricity negotiation [5]. These concerns have led to the adoption and development of competitive electricity markets. The rules and policies of electricity markets are very different worldwide, reflecting, in many cases, the reality of power systems in different regions [1]. Due to the significant changes in the power systems both in the production and in new usages of electricity, the markets and the power systems control itself are evolving in order to meet the present and future needs. New players and energy resources are emerging—notably electric vehicles (EV) [6], consumption flexibility and demand response (DR) [7], large penetration of renewable based generation [8], energy efficiency measures [9], and building energy management parties [10], among many others. Several types of aggregator players, taking advantage on concepts such as the smart grid and microgrid [11], as discussed in Section 2 are also emerging in order to boost the potential of smaller players and resources. This new paradigm in power systems has increased the complexity to manage and operate the transmission and distribution networks and the interactions between the traditional and new players. The uncertainties associated with renewable based generation, electricity market prices, energy consumption, or electrical vehicles trips are just a few examples of the increased sources of complexity brought to the power and energy sector. Now that the planning and operation must consider the small resources of new players, rather than just looking at the overall picture of overall consumption and generation, makes the problem-solving in this domain an increasingly complex task. To address this complexity, new approaches have been proposed in the research field, for instance, considering hierarchical management with the inclusion of increased intelligence at each level of the hierarchical control [12]. In such approaches, the main idea is to have a good balance between the complexity of the problems to be solved at each level and the number of iterations and quantity of information needed to exchange between each level of the hierarchy. However, multiple alternative solutions should take into account the reality of each region.
The development of new algorithms using advanced optimization techniques and artificial intelligence are being proposed and tested to enable dealing with the increased complexity allowing a cyber-physical representation of the reality [13]. These two worlds, the real and the virtual, should run in parallel, considering and learning from past experiences to forecast and estimate the future. The new large-scale and complex models that are required to model the power and energy system under the new paradigm cannot be solved in due time without the support of new intelligent simulation and optimization approaches.
This paper provides a survey on the latest advances on optimization and simulation in the new paradigm of power systems. The main focus is centered on (i) the optimization of power systems under the new smart grid context, analyzing energy resource management models that deal with the increased complexity brought by the large number of new players and energy resources in this domain, and especially by the uncertainty associated to the variability and fluctuation of prices, generation, and consumption. Optimization models including stochastic modeling, robust analysis, and innovative business models are discussed and compared; (ii) the discussion and analysis of the application of computational intelligence approaches as means to solve the complex optimization problems, while guaranteeing an adequate balance between the execution time, the use of computational resources, and the quality of the achieved results. Evolutionary computation techniques are discussed in detail, as one of the most widely used approaches to solve the complex optimization models; (iii) the analysis of large-scale simulation of electricity markets and smart grid. Multiagent-based approaches are especially relevant in this domain, which includes the interaction between purely virtual environments, and hybrid approaches combining software agents and physical resources.
This paper is organized as follows. After this introduction, Section 2 introduces and explains the main concepts that are used throughout the paper. Section 3 presents a general overview of the recent research work related to new arisen challenges in the scope of smart grid and power systems optimization. In addition, a critic analysis is given to research work on energy resource scheduling, seen as one of the most complex and challenging research problems faced in power systems optimization. Section 4 provides an overview of the application of computational intelligence techniques for solving complex optimization problems in the energy domain. An extensive analysis of evolutionary computation applications for optimization in the energy domain is provided, followed by a brief review on artificial neural networks and fuzzy systems applications. Section 5 reviews the current state of the art in complex large-scale simulation in power and energy systems. The section emphasizes two main topics, namely, electricity market simulation and smart grid simulation, and covers both simulation models and studies proposed in the literature, as well as large-scale simulators resulting from relevant international research projects. Finally, the conclusion provided in Section 6 summarizes the most relevant points identified throughout the document, including the main advances in the literature and the main limitations that lead to new emerging research paths.
2. Preliminaries
Power systems are one of the backbones of our modern society. In fact, the electric grid is considered one of the greatest inventions of the 20th century. Despite the big challenge that the generation, control, and management of electricity represent, speaking in an elevated level of abstraction, the process from the generation of electricity to the delivery of it to end-user was somehow simple to understand. For instance, Figure 1 shows a simplified design of electric grids. In the early days of the electric grid, the flow of electricity was well defined from generation facilities, traveling through the transmission and distribution grid, until reaching end-users. Simple communication capabilities were required for control and monitoring. So, despite that some of the processes inside the electric grid were complex, looking at the electric grid as a whole, and compared with the electric grid of today, one could say that its design presents low complexity and it is easy to understand.
The truth is that without the electric infrastructure, it would be hard to imagine our days. However, nowadays due to recent reforms in the energy sector, unprecedented changes have been observed in this field, most notably the proliferation of renewable energy sources, mainly wind and solar generation and increasing penetration of EVs [14]. In this context, we are witnessing the appearance of new terms in the power systems arena, such as smart grid and microgrid, which have distinct meaning. Although the definition of those terms is still evolving, it is accepted that smart grid refers to the merge of advanced computing technologies and communications with the electricity network. Equipment such as smart meters and phasor measurement units will contribute to an increase in observability and improvement in decision-making and situational awareness. Ultimately, the goal of smart grid is to provide a safe, reliable, and flexible grid operation, which is able to accommodate large number of renewables and other resources without sacrificing the reliability and cost of grid operation [15, 16]. On the other hand, microgrid represents a group of interconnected loads and generation resources with defined grid boundaries that can be controlled as a single entity. A microgrid in its pure definition should be able to operate isolated from the main grid, that is, in islanded mode. A university campus or a residential building can constitute an independent microgrid if the above definition is met. Since the microgrid relies on advanced computation technologies, it is possible to generalize that a microgrid is “smart” or part of a larger smart grid.
In this paper, we refer to the abovementioned changes as the new paradigm of power systems. On the one hand, these changes are contributing to reduction in the carbon footprint and increase in sustainability [17, 18]; on the other hand, power systems are becoming more complex and difficult to understand [19–22]. For instance, the electric grid from Figure 1 is evolving to something like the smart grid shown in Figure 2. The complexity of the smart grid is attributed to multiple players that interact and operate with independent behaviors and goals in a physical constrained network. These decisions impact the physical layer, and the overall result is difficult to be inferred from analyzing individual behavior. Moreover, no single entity can control, monitor, and manage in real time, which furthers contributes to higher system complexity.
If those new considerations were not enough, the technical constraints in modeling power systems are increasingly complex due to new societal challenges such as pollution and health impacts, ecological changes, multiple uses of land and water resources, and safety. These constraints are specific and translate into specific mathematical equations in optimization problems, making them harder to solve [23]. The current framework of smart grid involves several layers from the component parts (grid lines, equipment, and resources), including communication, management, and business layers.
In addition, smart grid design and implementation need to embed broader social and cultural considerations in order for smart grids to be successful. Not only do smart grids need to be understood as complex techno-socio-economic systems with multiple physical, cyber, social, policy, and decision-making layers; also, the interaction of those layers with changing external conditions (economic cycles, technological innovation, and prevailing and changing weather and climatic conditions) needs to be properly studied.
3. Recent Research to Tackle Challenges in the New Paradigm of Power Systems
3.1. Overview and Status of Recent Research
In the last years, research has been devoted to deal with this complexity and overcome the challenges brought by the new paradigm of modern power systems. We have looked at scientific research over the last few years (since 2010) in the SCOPUS database [24] (i.e., in December 2017) using different search terms to analyze the number of publications produced and provide an overview of the recent research. Table 1 shows the overview of research documents produced since 2010 (excluding 2018) analyzed by different search terms. The search terms have been combined with similar variants within the power systems field to observe results as accurately as possible. We looked for each search term in the abstract, title, and paper’s keywords. In addition, we investigate the most cited articles in each search criterion as well as the most noticeable research authors by the number of articles produced. A total of 23,619 documents between 2010 and 2017 are found in the SCOPUS database by using search term “smart grid” OR “smart distribution network” alone in the abstract, title, and keywords and 17,757 if the search term is “renewable generation” OR “renewable energy sources.” The search term “smart grid” AND “electric vehicle” produced the least number of documents when compared with the other search terms. In spite of their complementary, EVs in the smart grid context are yet a growing research field not as mature as the research observed in other topics, such as DR.
The most cited article we came across while looking at the produced research documents is [25] (3057 citations). The interest in grid storage applications is noticeable and considered the holy grail of modern power systems, since it can solve many problems and challenges arisen in smart grid. In particular, Dunn et al. [25] review battery systems for grid applications including redox flow, sodium-sulfur, and lithium batteries. The interest of storage for grid applications is related to diversified factors, including management of peak demands and grid reliability, postponement of grid investments, and integration of renewable energy sources.
Regarding “smart grid” OR “smart distribution network” term, the most cited articles we found are [11, 26, 27]. In [26] (1393 citations), an important advance is proposed in order to allow smartness and flexibility of smart grid based on ISA-95 standards. The system is more flexible and expandable using the proposed control. Farhangi [11] (1186 citations) highlights ingredients, drivers, and standards for the success and path of smart grid. Cabana et al. [27] (1144 citations) review the literature concerned with the materials used in lithium batteries, strong enablers of smart grids, and renewable technologies.
In “Renewable generation” OR “renewable energy sources” research topic, we could conclude that the 3 most cited articles are [25, 28, 29]. Blaabjerg et al. [28] (2256 citations) give an overview of the structures of distributed generation (DG) renewable sources based on fuel cell, photovoltaic (PV), and wind turbines. Carrasco et al. [29] (1989 citations) present a review of the appropriate storage system technology used for the integration of intermittent renewable energy sources. New trends in power electronics for the integration of wind and PV generators are presented.
In “smart grid” AND “electric vehicle” research topic, the most cited articles are [30–32]. Ipakchi and Albuyeh [30] (905 citations) provide an overview of the future of the power grid, namely, the necessary transformations to accommodate large number of EVs and other resources such as DR. Kamaya et al. and Skyllas-Kazacos et al. [31, 32] (856 and 574 citations) are related to this topic since they present progress related to lithium and flow batteries, which are fundamental and intrinsically related to the development and success of EVs. Lower battery costs and higher energy density are fundamental to leverage attractiveness and competitiveness of electric cars.
Regarding “demand response” OR “demand-side management” research topic, the 3 most cited articles are [30, 33, 34]. Mohsenian-Rad et al. [33] (1097 citations) propose a distributed demand-side management maintaining user privacy and details of energy consumption. The highly cited work emphasizes that the proposed approach may reduce the peak-to-average ratio of the total demand, the total costs, and each user’s individual daily electricity charges. Palensky and Dietrich [34] (830 citations) provide an outlook and taxonomy to demand-side management and several successful demonstration projects that have been implemented until the time of publication.
In the topic of “smart grid” AND “optimization,” the results in SCOPUS revealed that [33, 35, 36] are the most cited articles. Cheng and Chen [35] (836 citations) provide an outlook at the current state of the art in smart grid communications and open research issues in the field of information and communication technology (ICT). ICT is a key part of the successful deployment of optimization applications in the smart grid. In the search term “energy resource management” OR “energy management system”, [1, 11, 33] are the most cited works. Siano [1] (505 citations) provides a comprehensive survey on energy management, including DR potentials and benefits in smart grids. Figure 3 depicts the number of documents produced between 2010 and 2016 by search term according to the same terms previously depicted and analyzed in Table 1. It can be observed that the increase has been significant and well supported since 2010 when analyzing most of the search topics, except from the search term “smart grid” AND “electric vehicle.” There is substantial research work on EVs, which is not directly related to the “smart grid” topics (cf. [37]). If compared to “demand response” or other search terms, the number of documents suggests a much more active and mature development. This analysis also suggests that research on EVs in the smart grid context is yet in the early stage. Although many theoretical concepts and ideas have been proposed (e.g., dynamic pricing [38, 39] and vehicle-to-grid [6, 40]), both EVs and smart grid are yet in the early stage of introduction and lessons have to be learned to better understand how EVs can realize the full smart grid potential. Energy resource management-related research has increased steadily along with the research in DR topics since its importance to the new paradigm of power systems (see Section 3.2). In fact, energy resource management is key to smart grid operation in order to attain acceptable costs with reduced impact from uncertainty of intermittent generation and load sources of difficult anticipation such as EVs.
Figure 4 shows the number of conference papers and journal articles produced between 2010 and 2016 (primary axis) by search term “energy resource management” OR “energy management system” in the SCOPUS database. In the secondary axis, the conference papers per journal article indicator are shown (green cross). Other publications such as book chapters are not included in this comparison. The number of journal articles produced per year has increased substantially from 89 in 2010 to 428 in 2016 (more than 4-fold increase). However, the number of conference papers per journal article has reduced significantly in the last three years to 1.27 from more than 2.23 in 2011 to 1.27 in 2016. This suggests that more journal articles are being published for each paper presented at an international conference. In our opinion, this is a potential indicator that energy resource management-related research is getting more mature in the last years and ready for publication in journal.
3.2. Review of Works in the Energy Resource Management
Energy resource management has been considered by some researchers as one of the most complicated optimization problems in power systems due to its combinatorial nature, nonlinearities, and large number of energy resources which leads to high dimensionality and highly constrained problems [23, 41–43]. Adequate optimization frameworks are a key part of the new paradigm brought by the smart grid operation. The literature has a relevant amount of publications in this field that we selected and revised in this section. Since consideration of uncertainty is crucial for smart grid models [44], we investigated in detail this component in the studied models. We also include papers published in 2017 (55 publications appear in the SCOPUS database with the search term “energy resource management” OR “energy scheduling” AND (“uncertainty” OR “uncertain” OR “variability”) between 2010 and 2017, of which 22 are from 2017). However, we could observe that the majority of the proposals still lack the consideration of full component uncertainty and/or DR in the model. Table 2 identifies the characteristics of the main publications we selected in what regards the considered energy resources and in terms of uncertainty consideration in the model.
As can be observed in Table 2, very few works attempt to consider most sources of uncertainty in a joint energy scheduling model [45, 46]. Moreover, it is yet not common to see works that incorporate vehicle-to-grid (V2G), DG, DR, and energy storage systems (ESS) simultaneously as in [45–48]. However, [47, 48] do not include consideration of uncertainty of the energy resources.
The work developed in [49] approaches the problem of day-ahead electricity market bidding to minimize charging costs and satisfy EV demand of the aggregator. The work considers the presence of uncertainty, but V2G is not considered in the problem. Results suggest that controlling EV charging is important to lower energy costs, while the aggregator has limited market potential with reasonable amount of EVs. A related work can be seen in [50] where a bidding and power scheduling is proposed to maximize the expected benefit of a microgrid. The stochastic model captures several sources of uncertainty, and the novelty lies in the exploitation of thermal characteristics of the buildings to mitigate renewable imbalances. Also, works [51–53] propose to tackle the optimal bidding in the market of the aggregator using multistage stochastic [51, 53] and robust approaches [52]. The work in [54] introduces the reliability of the fuel cell outages in the microgrid energy scheduling, but the uncertainty considered in the model lies only in the fuel cell outages.
The model in [55] proposes a scheduling model for microgrids under uncertainty. The work implements a two-stage stochastic model with one external power flow algorithm to calculate power losses. EV uncertainty, DR, and V2G are not considered though. To overcome this, [45] propose an improved two-stage stochastic model that addresses several sources of uncertainty, namely, wind, PV, EVs, demand, and market price, in a joint model yet not accounting for power losses. Later, a new model is proposed in [46], which adds network constraints, namely, power line capacity and voltage control to the original problem in [45]. The problem is solved using a Benders decomposition scheme. The results suggest that the large-scale problem with uncertainty can be solved in its most complex form, dealing at the same time with the number of resources and the DSO technical constraints as part of the equation.
The two-stage stochastic model introduced in [56] adds to the previous stochastic microgrid energy scheduling works, a hybrid AC/DC microgrid setting. However, the work does not consider DR application. The model in [57] proposes a day-ahead energy and reserve scheduling for the aggregator. The uncertainties are modeled via a point estimate method. The work does not consider EVs as an intelligent load. The work in [58] proposes a similar work but considering compressed air ESS. EVs are not considered as well.
Most of these works have implemented a two-stage stochastic model, which requires knowledge about the distribution of the uncertain data, usually possible if historical records are available. However, it is fair to recognize that most of these models require large amounts of computational resources to be able to solve the stochastic model with an adequate number of scenarios (even if scenario reduction techniques are adopted). In fact, scenario reduction techniques decrease the accuracy of the uncertainty representation. Since smart grid operation is dealing with an increasing number of energy resources and consequently more components associated with uncertainty, it remains a major challenge to tackle these optimization models under uncertainty with adequate representation. A solution may lie in metaheuristics and decomposition techniques combined with uncertainty models and/or robust optimization models that deal with a range of uncertainties instead of probabilistic scenarios. Another solution may lie in distributed real-time energy scheduling as proposed in [59], which transforms the problem into a distributed and tractable problem, using rolling horizon optimization and Gaussian approximation. However, we believe the work is still limited since it considers only a single energy source and one source of uncertainty.
Alternatively to stochastic models, Ju et al. [60] propose a robust model for the energy aggregator with DR, ESS, and renewables, which does not require to know the distribution of uncertain parameters such as the forecasts. Still, EVs are not considered in the work as well. Another robust model is adopted in [61] which tackles market price and load demand uncertainty for aggregators that are new entrants to the market and have little knowledge on the behavior of the market and their customers. In [48], authors propose an energy resource management for domestic loads that considers uncertainty in the PV power for the day-ahead and real-time approach. The work considers EVs, DR as flexible loads, and ESS units. Two different stochastic methods are compared and evaluated using a realistic case study.
Table 3 classifies the works reported regarding its main purpose, namely, technical, economic, and environmental aspects. It can be seen that most of the works related to uncertainty deal with economic aspects. Technical aspects are often common and related to power losses and voltage control (when network constraints are considered) such as in [49, 56]. Some of the works consider environmental, reliability, and building dynamic aspects such as in [50, 54]. Soares et al. and Soares et al. [46, 47] include both technical and economic aspects. However, [47] does not incorporate the resources’ uncertainty as formulated later in [46], despite not including environmental aspects as in [47].
4. Computational Intelligence for Complex Optimization Problem in the Energy Domain
Computational intelligence (CI) is usually referred to a family of problem-solving and problem-stating techniques that attempt to exhibit or mimic the “intelligence” observed in nature. It includes some of the most popular paradigms in the applications of computer science, namely, evolutionary computation (EC), artificial neural networks (ANN), and fuzzy systems (FS).
Despite its popularity nowadays, the term CI has had different definitions through the years in the scientific community and typically is seen as a separate area of artificial intelligence (AI) very related with soft computing or natural computation. For instance, the first CI definition is typically attributed to J.C. Bezdek who defined CI in 1992 as “Computational systems that depend on numerical data supplied by manufactured sensors and do not rely upon knowledge” [63]. Later, in 2002, Engelbrecht defined CI as “the study of adaptive mechanisms which enable or facilitate intelligent behavior in complex and changing environments” [64]. Therefore, in an abstract sense, CI can be defined as intelligent computational tools to solve, or model, complex problems. However, two main questions arise regarding this statement, namely, what is a complex problem? And more importantly, why should we use CI to address such issues?
To answer the first question, notice that many real-world problems are highly constrained by nature and face issues related to high dimensionality, lack of information, noisy and corrupted data, real-time requirements, and so on. Due to these characteristics, exact mathematical formulations sometimes fail in providing solutions because of either restriction of time, lack of memory to deal with large-scale problems, or other factors. In other words, problems that cannot be efficiently solved by traditional mathematical approaches can be seen as complex problems.
On the other hand, and answering the second question, CI is strictly related to the subfield of computer sciences known as soft computing. Soft computing (Soft computing became a formal area of study in computer science in the early 1990s and in a broader sense includes topics such as computation intelligence (CI), machine learning (ML), and probabilistic reasoning (PR). In this paper, however, we focus our attention to the main three subfields of CI; namely, an extensive analysis of EC applications for optimization in power systems is provided, with a brief review on optimization-related ANN and FS applications.) is used to solve complex problems (i.e., with characteristics such as the ones previously stated) in which exact methods cannot compute a solution in polynomial time. Different from hard computing, soft computing is tolerant to imprecision, uncertainty, and approximation. Even when soft computing cannot guarantee an optimal outcome, it can return near-optimal solutions in acceptable computational times and with low memory requirements, imitating the role model of the human mind.
It is important to point out that despite the conditions mentioned above, CI has demonstrated satisfactory performance in a wide variety of application fields such as power systems, portfolio optimization, transportation, smart grid, and telecommunications.
As you might notice, this section is devoted to the application of CI for solving the wide variety of complex problems that arise in the new paradigm of power systems. As it was stated in Section 2, current smart grids are evolving to a very fast pace, and the interaction between a considerable number of new participants and elements (with their inherent socioeconomic and technological aspects) makes the analysis of such systems very complex for traditional mathematical tools and management and control models. In addition, the same logic applies to different areas of engineering since CI is a paradigm that is acquiring significant importance these days.
4.1. Computational Intelligence Classification and Application Target
CI is commonly used for different purposes, but we can devise three main categories of its application, namely, optimization, learning/modeling, and control. Moreover, some problems require the use of a combination of these categories to devise practical solutions. Despite the vast synergies existent in the application targets of CI, such targets are strongly related with the three main paradigms of CI. For instance, evolutionary computation (EC) is applied to optimization problems; artificial neural networks (ANN) is extensively used for learning/modeling and forecasting, and fuzzy systems (FS) is typically implemented for control. The relation of EC, ANN, and FS, to the target applications is used to envisage application areas in a broader sense somehow. However, many real-world applications may require synergies between methods belonging to the three fields to obtain an acceptable solution.
With the evolution of technology in diverse engineering fields, real-world problems are becoming more challenging, limiting the use of exact methods. Due to this, CI has emerged as a powerful tool to provide acceptable solutions in different domains. As a proof in the increasing interest of the research community regarding CI applications, Figure 5 shows the number of publications in the main three topics of CI from 2000 to 2016 (17-year span), obtained after a search in the SCOPUS database [24]. The search was done putting as a filter the terms shown in Figure 5, and considering all areas of applications. As a result, 24,706 publications in total were found when searching for “evolutionary computation OR evolutionary algorithms,” 23,995 publications using “Artificial Neural Networks,” and 49,623 when searching for “Fuzzy systems.”
It is evident that CI is gaining interest mainly for its effectiveness to solve a large variety of problems in all areas of engineering. Moreover, from the total number of papers found in the three analyzed areas, 10% (2601 publications of EC), 22% (5299 publications of ANN), and 2% (1410 publications of FS) of the published works correspond to studies in the energy domain. To narrow our review into the topic of interest, that is, CI applied in the energy domain, we have filtered the search focusing on the publications regarding CI in the energy domain.
Figure 6 shows the ratio between the number of publications devoted to the energy domain and the total of publications by the CI paradigm (i.e., EC, ANN, and FS). It is worth noticing that the three areas of CI show a trend that reflects more publications in the energy domain. From the three main topics of CI, EC is the one that shows the most pronounced increment in the ratio in the last years.
In the following subsections of Section 4, we include a brief analysis of the taxonomy of EC, and through Tables 4–7, we group the most cited works of the CI paradigm applied to the energy domain. The tables include the CI techniques, the problem to be solved in the energy domain, and the main mathematical characteristics that make such problems complex and justify the use of advance computational approaches such as CI.
4.2. Evolutionary Computation (EC) Taxonomy
Evolutionary computation (EC) is a fundamental part of CI and encompasses a set of algorithms for global optimization mostly inspired by biological and evolutionary processes. CI is arguably one of the most successful branches of CI, used by a large number of practitioners all over the world and in all areas of engineering [65] (see Figures 5 and 6). It is worth noticing that EC is not only limited to the study of evolutionary algorithms (EA), as it is sometimes wrongly assumed, but also covers some of the most exciting trends in computer sciences, namely, swarm intelligence (SI), nature-inspired algorithms, and bioinspired or natural computation.
The family of algorithms belonging to the class of “evolutionary algorithms” (EA) shares some common characteristics that distinguish them from other metaheuristics. In technical terms, EAs are population-based problem-solvers. Commonly, EAs act over an initial set of candidate solutions (i.e., a population) that is iteratively updated through generations. The performance of the solutions is measured by a given fitness function, and at each generation, solutions with inferior performance are stochastically removed, whereas new solutions (generated through a specific operation particular for different EAs) are introduced into the population. As a result, it is expected that by the principles of natural selection (or artificial selection in this case), the population gradually will evolve towards the optimal fitness value.
The field of EAs is broad, and the techniques typically differ in implementation details or the genetic representation they use. Among the existing techniques of EA, the three main subfields (source of inspiration for many developments in this paradigm) correspond to genetic algorithms (GA) [66, 67], genetic/evolutionary programming (GP) [68, 69], and evolutionary strategies (ES) [70, 71]. In the last decades, differential evolution (DE) [72], a particular case of ES, has been the subject of an entire line of research. DE is arguably one of the most popular and effective EAs with a considerable number of successful applications in the engineering domain [131].
On the other hand, SI and nature/bioinspired computation are metaheuristics inspired in the observation of the collective behavior of agents or natural/biological processes, respectively. For instance, SI takes advantage of the interaction of simple agents with each other on the environment to achieve collectively “intelligence,” similarly to natural systems such as ant colonies or bird flocking. Examples of SI algorithms are the ant colony optimization (ACO) [73], particle swarm optimization (PSO) [74], artificial bee colony (ABC) [75], or cuckoo search (CS) [76]. Nature-inspired algorithms go further in the design of algorithms inspired by natural, physical, or even chemical phenomena. Examples of this type of algorithms are gravitational search algorithms (GSA), simulate annealing (SA), harmony search (HS), and others. A very well-conducted taxonomy and classification of nature-inspired algorithms can be found in [77].
4.3. Evolutionary Computation (EC) in the Energy Domain
EC can be applied to problems of all areas of engineering. In this review, however, we are interested in applications related to complex problems in the energy domain. Therefore, in Table 4, we have grouped the most cited works of the EC paradigm applied to the energy domain in the span 2010–2017. The table is organized from the top 1 cited paper to the top 10 according to the SCOPUS database. We show the specific EC approach used, the problem that was solved, and some characteristics related to the work.
In general terms, Table 4 confirms that EC is mainly used to solve optimization problems. Problems such as economic load dispatch (ELD), optimal power flow (OPF), and optimal operation of microgrids are addressed by the most influential works, mainly due to the key role that those problems play in power systems. With the evolution of the electrical grid led by the penetration of distributed resources, and the deployment of smart grid technologies, EC has attracted the attention of many practitioners due to its effectiveness in providing acceptable solutions when exact traditional methods fail. It is also interesting to appreciate the correspondence between some of the most studied algorithms in the field of EA and the most cited papers of EC in the energy domain. For instance, we can find the application of the strength Pareto evolutionary algorithm (SPEA) [78], an algorithm that has been cited more than 6000 times according Google Scholar, into three problems of the energy domain, namely, wind turbine placement, turbine layout design, and the optimization of a stand-alone PV-wind-diesel system with battery storage. The nondominated sorting genetic algorithm NSGA-II [79], another well-known and highly used EA, was applied to the combined cycle power plant (CCPP) optimization problem in [80]. DE, well-known for its simplicity and effectiveness in continuous global optimization, has been used to solve the OPF problem [81] and the optimization of PV systems [82].
In general, Table 4 shows that the majority of the problems in which EC is applied corresponds to complex optimization problems that cannot be solved efficiently by traditional techniques. Such complex problems present common characteristics, namely, multiobjective nature, constrained optimization, convex and nonconvex problems, nonlinear formulations, and so on. Because of the advances in technology, it is expected that the complexity of the systems will increase in the years to come, and therefore, the complexity of the mathematical formulations required to solve related problems is expected to grow as well. In this likely scenario, EC seems a viable and logic option for application into the field, despite the fact that optimal solutions cannot be guaranteeing.
The search of the most cited papers was done in the time span 2010–2017. However, the top ten cited papers correspond only to the span 2010–2013. To give a glimpse into more recent works and envisage the research directions of the application of EC nowadays, Table 5 includes the three most cited papers of the years 2015, 2016, and 2017.
In general aspects, it can be appreciated in Table 5 that the tendency of recent influential works lays on the hybridization of EC algorithms. Combinations of DE and PSO into DEPSO [90], ACO and ABC into ACO-ABC [91], or a multistep approach using cuckoo search (CS) algorithm, fuzzy system (FS), weather research and forecasting (WRF), and ensemble forecast (CS-FS-WRF-E) [92] are some examples of the synergies that can be produced between EC algorithms for solving more challenging problems.
On the other hand, in the recent applications of EC, we still can find the application of pure popular EC algorithms. For instance, DE was recently applied to the optimization of combined cooling, heating, and power-based compressed air energy storage. A GA was used in [93] to determine optimal sizing of components in microgrids, while MILP was then used to solve the unit commitment problem (UCP), which is considered of paramount importance in the management and operation of microgrids. NSGA-II was used in [94] to solve a multiobjective optimization problem of solar driven Stirling heat engine with regenerative heat losses. In addition, new algorithms are under development, and their application is also very welcome in the energy domain. Brainstorm optimization algorithm (BSOA) and chaotic bat algorithm (CBA) [95] are only two examples of successful applications of such new algorithms. BSOA was recently applied to the optimal location and setting of flexible AC transmission system (FACTS) devices in [96], whereas CBA was used to solve the well-known economic dispatch problem (EDP) considering equality and inequality constraints [97].
4.4. Artificial Neural Networks (ANN) and Fuzzy Systems (FS) as Complementary Tools for Optimization
ANN and FS are two important paradigms of CI used mainly for forecasting/modeling and control, respectively. Since the scope of this review is on complex optimization and simulation, the extensive analysis of the applications of ANN and FS is somehow out of the scope of what this review is intended for. However, these two paradigms are also widely used in combination with EC for solving some optimization problems. Moreover, some applications of ANN or FS, such as the ANN weight selection or the definition of the parameter of membership functions of an FS, are indeed highly complex optimization problems that cannot be solved with deterministic methods suitably. Therefore, this section revises some of the hybrid applications in which ANN, FS, and EC are used in combination to solve complex tasks.
The pure application of ANN in power systems is commonly used to solve forecasting problems. Forecasting of solar irradiance or PV power [100–103], wind speed [104–106], or load [107] are just some examples of successful applications of ANN in the energy domain. On the other hand, FS are more commonly applied as operation and control strategies. For instance, in [108], a fuzzy logic controller (FLC) was designed for operation and control of an offshore wind farm interconnected to a high-voltage DC (HVDC) system. The maximum power point tracking (MPPT) for PV, a problem sometimes solved by ANN, was addressed using FLCs in [109, 110].
Nevertheless, ANN and FS are also commonly coupled with EC, giving, as a result, some hybrid approaches that attract the attention of the research community to a significant extent. In an effort to synthesize such pieces of work, in Table 6 we have selected some of the most cited works regarding hybrid models using ANN and FS in the span 2010–2017. A combination of fuzzy inference systems, wavelet transform, and PSO is used [111, 112] for forecasting. In this case, the definition of the parameters associated with the membership functions of the fuzzy inference system is a complex optimization problem, and therefore, PSO is used to this end, obtaining more accurate results. On the contrary, in [113], a fuzzy system is used to tune the parameters of PSO, giving place to the so-called fuzzy adaptive modified particle swarm optimization (FAMPSO), which is used for solving the nonconvex economic dispatch problem (NEDP). These two examples show how the interactions between ANN, FS, and EC might go into different directions, giving place to the envisage of more refined techniques when a good knowledge of the strengths and weaknesses of the involved approaches exists.
We have also selected some recent hybrid approaches corresponding to highly cited papers related to ANN and FS applications. Table 7 presents such selection, and it is interesting to notice that the majority of the selected work is related to forecasting techniques [92, 102, 107, 114–116]. This trend is highly motivated by the necessity of a more efficient integration of renewable generation into existing electrical grids. Since forecasting techniques will never guarantee a perfect match between realizations and estimations (or at least we seem to be far from that goal), the improvement of existing techniques is of paramount importance in the operation of power systems.
Overall, the revised literature shows that CI has a strong potential for solving complex tasks in the energy domain. The use of hybrid techniques, along with some other powerful available resources, such as super computers or parallelism, opens a window of opportunity to refine existing techniques and make them more accurate and reliable tools.
5. Complex Large-Scale Simulation in Power and Energy Systems
5.1. Electricity Market Simulation
Renewable energy sources such as wind and solar variable and intermittent nature pose new challenges to the power sector and also to electricity markets. Many different market approaches have been experimented all around the world, and all have been subject to multiple revisions. The primary focus is on adapting electricity markets to deliver their intended economic efficiency and reliability outcomes under the new paradigm of growing share of renewable energy sources [118]. One of the main European Union (EU) priorities concerns the formation of a pan-European energy market. The majority of European countries have already joined together into common market operators, resulting in joint regional electricity markets composed of several countries. Additionally, in early 2015, several of these regional European electricity markets have been coupled in a common market platform, operating on a day-ahead basis [119]. That achievement has been enabled by the multiregional coupling (MRC), a pan-European initiative dedicated to the integration of power spot markets in Europe. The common market platform has resulted from an initiative of seven European power exchanges, called price coupling of regions (PCR) [119], which have joined efforts to develop a single price coupling solution used to calculate electricity prices across Europe and allocate cross-border capacity on a day-ahead basis. This is a crucial step to achieve the overall EU target of a harmonized European electricity market.
This type of initiative is being accommodated by the European Commission, which has created the basis for a significant number of European projects that have been giving substantial contributions to deal with some of the most prominent issues in the field, as discussed in [120] and summarized as follows. One of the main hurdles for the operation of a pan-European market concerns the European power network constraints. The e-Highway2050 project aims at developing and applying a methodology for the long-term development of the pan-European transmission network able to ensure a reliable power delivery from renewable sources, and their integration in the pan-European market. The Optimate (An Open Simulation Platform to Test Integration in MArkeT design of massive intermittent Energy) project had a simulation platform as output, which accommodates the simulation of the pan-European electricity market. In the scope of CASSANDRA (a multivariate platform for assessing the impact of strategic decisions in electrical power systems), the participation of small consumers in the electricity market has been modeled, including the assessment of their strategic decisions. eBADGE (Development of Novel ICT tools for integrated Balancing Market Enabling Aggregated DR and DG Capacity) is developing a simulation and modeling tool for integrated balancing and reserve markets. EMELIE (electricity market liberalisation in Europe) focused on the liberalisation of the European electricity market and its potential impacts on market developments; this project has resulted in a modeling and decision support tool for analysis of various strategic behaviors in the scope of the European electricity market. E-PRICE (Price-based Control of Electrical Power Systems) approached the issue of errors in the prediction of both production and demand and their impact on ancillary services and reserve capacity. Renewable portfolio addresses market players’ optimal scheduling under integrating renewables in EU electricity markets, considering a stochastic dynamic market environment.
These initiatives are, however, strongly directed to the perspective of market operators and regulators, which results in valuable advances in what concerns market mechanisms, and market operation and validation, but, apart from a few exceptions, almost entirely disregard the decision support of market participant players, and the impact that the interaction between these entities has on the market. This decision support is, however, essential, since the complexity and the constantly changing environment of electricity markets makes the decision-making process of the involved entities a very difficult task. Operators and regulators need to foresee market developments and experiment and test new market rules and mechanisms. Market players need to understand how and when to participate in each market.
Due to the rising needs, several electricity market simulators have been developed, as can be consulted in [5, 121–124], which provide interesting reviews on electricity market simulators. In fact, a total of 2454 documents related to electricity market simulation can be found in the SCOPUS database, until the end of 2017. Such works start emerging slowing from 1981, and several peaks of interest in this theme are well highlighted throughout the years, as can be seen from Figure 7.
As can be seen from Figure 7, there has been a significant increase in the interest in research on electricity market simulation in the beginning of the millennium, with a huge increase in the number of documents published during these few years. In fact, the 10 most cited documents in this area all refer to the first decade of the 2000s, as depicted in Table 8. An overview of the most cited documents and top authors related to electricity market simulation is shown in Table 8. The tendency keeps on increasing from that point onwards, but there are several obvious peaks of interest, which are closely related to the most relevant changes in electricity market policies, as well as market model restructuring all around the globe.
Some relevant examples of electricity market simulators resulting from these works are [5, 122] the Simulator for Electric Power Industry Agents (SEPIA) [135], which is a Microsoft Windows-oriented platform; Power Web [136], a Web-based market simulator that allows participants to interact from very distinct zones of the globe; and The short-medium run electricity market simulator (SREMS) [137], which is based on game theory and is able to support scenario analysis in the short-medium term and to evaluate market power. These simulators present a common limitation, as they lack flexibility in order to deal with dynamic environments with complex interactions between the involved entities. Multiagent systems are computer systems composed of autonomous agents that interact to solve problems beyond the individual capabilities of each agent [138]. In the last decade, a number of multiagent electricity market simulators have emerged [124, 139].
An agent-based simulation model of the England and Wales electricity market is presented in [140]. The simulation model enables comparing alternative market mechanisms, including different settlement systems and bidding periods. The same model is applied to the German electricity market in [141]. The proposed simulation model defines agents through several attributes. Reinforcement learning is used to support agents’ decisions. The analysis of alternative auction models is an important asset of this simulator; however, the complete functionalities of electricity markets are not modeled, and market participation is restricted to producers [142]. A wholesale market simulation model has also been developed by the MIT [143]. In this simulator, a static number of three generator agents use the derivative following strategy [144] to learn the best bids to apply in order to maximize the profits. The market model is very simple, with only generators being able to act dynamically. These agents can bid their complete generation capacity or just a part depending on the expected profits [145]. A simulated market model based on double auction is presented in [146, 147]. The market model matches consumers and generator bids pairwise [148]. The focus of this simulator is on the learning process of agents’ bidding behavior, with the application of the Roth-Erev reinforcement learning algorithm [147] and genetic algorithms [146]. An electricity market simulation study that compares three alternative pricing methods in electricity market auctions is presented in [149]. Agents in this simulation model submit bids for the entire generation capacity, and they learn to bid mark-ups on top of their marginal costs. A model consisting in three sequential energy markets representing, namely, a gas market, a wholesale electricity market, and a retail electricity market, is presented in [150]. All these market models are modeled as uniform-price auctions, in which only supply agents submit their bids. The focus is on the study and analysis of the dynamics of reward interdependence in the different markets [151]. An electricity market simulation study directed to Nash equilibrium analysis is presented in [152]. Q-Learning is used to provide learning capabilities to the involved agents, considering a market pool considering transmission grid constraints. Social welfare implications are analyzed for several congestion management approaches. A multiagent simulation system to model the UK electricity market is introduced in [153]. This simulator includes several different pricing calculation schemes, and the focus of the study is on producer agents’ learning behavior. The role of other relevant involved agents, such as consumers or the independent system operator (ISO), is, however, not considered. Additionally, the simulated market model represents the UK electricity market before its restructuring process of 2001; hence, this model is severely outdated.
The Genoa Artificial Power Exchange (GAPEX) [154] is an agent-based framework for modeling and simulating power exchanges. GAPEX is implemented in MATLAB and allows the creation of artificial power exchanges reproducing exact market clearing procedures of the most important European power exchanges [155]. The Electricity Market Complex Adaptive System (EMCAS) [156] uses an agent-based approach with agents’ strategies based on learning and adaptation. Different agents are used to capture the restructured market heterogeneity, including generation, demand, transmission and distribution companies, independent system operators, consumers, and regulators. It allows undertaking electricity market simulations in a time continuum ranging from hours to decades, including several pool and bilateral contract markets [157, 158]. Agent-based modeling of electricity systems (AMES) [159] is an open-source computational laboratory for the experimental study of wholesale power markets restructured in accordance with US Federal Energy Regulatory Commission (FERC)’s market design. It uses an agent-based test bed strategically learning electric power traders to experimentally test the extent to which commonly used seller market power and market efficiency measures are informative for restructured wholesale power markets. The wholesale power market includes an independent system operator, load-serving entities, and generation companies, distributed across the busses of the transmission grid. Each generation company agent uses stochastic reinforcement learning to update the action choice probabilities currently assigned to the supply offers in its action domain.
The Multiagent Simulator of Competitive Electricity Markets (MASCEM) is a simulator developed by the Polytechnic of Porto, Portugal [160, 161]. MASCEM supports the simulation of several market models, including the simulation of the market models of MIBEL (Iberian market), EPEX (central EU market), and Nord Pool (northern European market), as well as some other specific market models, for example, CAISO’s. The network validation is also possible, although in a simplistic manner. Agents in MASCEM are endowed with machine learning and decision support capabilities through its integration with the Adaptive Decision Support for Electricity Markets Negotiations (AiD-EM) system [162]. The participation of DG, small consumers, DR, and EVs, among other resources, is enabled through the connection of MASCEM with the Multiagent Smart Grid Platform (MASGriP) [163]. The integration between the several systems is achieved through the use of ontologies to support the communications between the agents of the different systems. The National Electricity Market Simulation System (NEMSIM) is an agent-based electricity market simulator developed specifically for simulating Australia’s electricity market. The agents considered in NEMSIM are electricity producers, network service providers, retailers, and the National Electricity Market Management Company (NEMMCO) [164]. The market models in NEMSIM include bidding and bilateral contracts. The technical constraints of the transmission network are considered, even if only in a superficial way. Negotiating agents have independent learning capabilities, based on the agents’ specific goals. However, since NEMSIM is designed particularly for the Australian market, its model is too specific to be easily adapted to other market models. Another simulation model that follows the Australian market rules is presented in [165]. This model includes two bidding mechanisms, namely, stepwise and piecewise linear bidding. Simulations are restricted to a couple of generators in order to study the learning process of tacitly collusive strategies [166].
The short-term electricity market simulator-real time (STEMS-RT) has been developed by the Electric Power Research Institute (EPRI) [167, 168]. STEMS-RT models the market and accommodates both human participants and computer agents. Human and simulated agents represent buyers and sellers and are able to set up and submit their bids. Two autonomous biding strategies are available for suppliers, namely, a conservative approach, consisting in bidding all generations at the marginal cost. The second approach tries to maximize the profit on a short-term basis [169]. STEMS-RT, however, does not include some important electricity market participants such as consumers, aggregators, or the ISO. Also, STEMS-RT is restricted to the bidding process in the pool market, and although it provides two autonomous bidding strategies, there is no actual learning process for agents’ decision support. A two-settlement market composed by one forward market and one spot market is simulated in [170]. The agents present in this simulator are consumers, generators, transmission line provider agents, and the ISO. The focus of simulations is put on the decision-making of generators and the ISO. The problem is formulated as an equilibrium problem with equilibrium constraints. The strategy for representing the decision-making and solving it mathematically is similar to that of STEMS-RT. PowerACE is an agent-based simulation tool developed to study the CO2 emission trading market [171]. PowerACE is composed by several types of agents, namely, producers, consumers, traders, long-term planners, market operators, certificate traders, load serving entities, and consumers. The market model includes a market for CO2 emission allowance that complements the usual pool and bilateral markets. The capability of PowerACE investigating environmental issues (such as CO2 emission) is an important asset in upcoming simulators or in the improvement of existing ones, as it can largely affect the long-term investment decisions of generation companies. A multiagent architecture for decision-making in decentralized electricity markets called EMMAS (Electricity Market Multiagent System) is presented in [172]. Agents are introduced as (i) basic (atomistic), which can be a consumer, generator, transmission system operator, distributor, market operator, wholesaler, retailer, or a regulator, and (ii) synthetic (combined), which are combinations of different basic agents. EMMAS agents are able to learn from past experiences or directly from the domain of expertise. Both the distribution and transport grids are modeled.
Electricity market simulation is an area that has been attracting a significant interest during the last years. Many of the existing simulators have been discontinued, while others have been adopted or purchased by governmental entities. Tracking of their recent developments has been proven to be a difficult task, as it is possible to observe by the publication year of many of the review works. Some further relevant works related to electricity market simulation can be consulted in [173], which presents an electricity market simulator for mainland Spain; [174] presents the Marketecture from Los Alamos National Laboratory; [175] details the N-ABLE™, the Agent-Based Laboratory for Economics developed at Sandia National Laboratory; and [176] presents some simulations for coupled systems using the GridWise™ system, at the Pacific Northwest National Laboratory. Table 9 provides a comparison of the main characteristics of the most relevant reviewed electricity market simulators.
From Table 9, it is visible that most of the existing simulators include some types of agent learning capabilities, although these are almost entirely exclusively directed to generation agents. Many of these simulators also consider grid constraints, making them suitable for simulation under operators’ and regulators’ perspective. However, the study of the integration of distributed renewable generation and integration of DR or even of aggregators in the market is still very poor. This deficiency makes most of these simulators still unable to provide a valuable support for the study of the evolution of the power system in accommodating the targeted large amounts of renewable generation and to incentivize the active participation from the consumers’ side.
5.2. Microgrid and Smart Grid Simulation
One of the main achievements of the power and energy sector in recent years is the common acceptance by the involved stakeholders that power systems require major changes to accommodate in an efficient and secure way an intensive use of renewable-based DG [177]. The conclusion that the so-called smart grids are required is a crucial foundation for the work to be done in the coming years towards the modernization and restructuring of the power sector according to the new paradigms [20]. Huge investments have already been made in projects concerning smart grids, including research and development projects, pilot installations, and rollout of smart metering. A list of a total of 950 projects related to smart grids starting from 2002 up until today, amounting to €5 billion investment, is published in [178]. The domains with the highest investment are smart network management, demand-side management, and integration of DG and storage, together accounting for around 80% of the total investment. The large number of smart grid-related projects is resulting in important advances in the field, namely, concerning demonstration pilots and management and control methodologies.
The European Technologic Platform Smart Grids (ETPSG) for future electric networks is the main European forum for the establishment of R&D policies for the intelligent electricity grid sector [179]. The mission of ETPSG is (i) to create and maintain a shared sight for the future of electricity networks in Europe and serve as a catalyzer for their implementation and (ii) promote research, development, demonstration, and implementation projects related to intelligent electricity networks [120].
Under the ETPSG and due to the increase in funding from the EU in this field, several large-scale European projects have been providing a substantial contribution to deal with some of the most prominent issues in the field, as discussed in [120]. The e-Highway2050 project aims at developing and applying a methodology for the long-term development of the pan-European transmission network, with the objective of guaranteeing a reliable power delivery from renewable sources, and their integration in the pan-European market. This project adopts a top-down methodology to support the planning from 2020 to 2050. In this context, the output of this work should potentiate the idealization of a possible super European power network, which interconnects the large part of European countries. IRENE-40 is a European project that aims at providing an Infrastructure Roadmap for Energy Networks in Europe. The objectives of IRENE-40 are to identify strategies for investors and regulators to build a more secure, ecologically sustainable, and competitive European electricity system, which are presented in a roadmap, that is, a timeline with actions and a description of development stages towards future electricity networks over the coming 40 years.
WILMAR (Wind Power Integration in Liberalised Electricity Markets) has focused on the study of the impact of large-scale penetration of wind-based generation and its accommodation in EM, while TRADEWIND—Wind Power Integration and Exchange in the Trans-European Power Market—led by EWEA (European Wind Energy Association) aims at facilitating the dismantling of barriers for the large-scale integration of wind energy in European power systems, on transnational and European levels, and to formulate recommendations for policy development, market rules, and interconnector allocation methods to support wind power integration. TRADEWIND explores the benefits that a European grid with better interconnections and an improved power market design can have on the integration of large amounts of wind power. EWIS—European Wind Integration Study—towards a successful integration of large-scale wind power into European electricity grids provided a market model that represents the idealized operation of existing day-ahead markets respecting declared cross-border transfer capabilities. It simulated daily actions of transmission system operators (TSOs) to redispatch generation to meet actual network physical limits and respond to emerging information concerning demand, wind output, and other generation changes [120].
IDE4L focuses on providing flexibility services and distributed control for aggregators. The role of the distribution system operator is the core entity of this project, addressing several distinct market models. PV-Prosumers4Grid is studying novel self-consumption and aggregation models for prosumers, with the aim of supporting large-scale PV integration in the system. EU-SysFlex aims at achieving an efficient coordinated use of flexibility to enable the wide spread of renewable generation. This requires defining the right amount of flexibility and system services to support transmission system operators. iDistributedPV is developing affordable solutions to increase the penetration of distributed solar PV based on the effective integration of solar PV equipment, including the exploration of several related concepts and approaches. The main goal is to enhance the role of the prosumer.
The significant work being developed in the scope of the multiple projects in the field is resulting in several relevant works in the domain of smart grid and microgrid simulation. In fact, an astonishing number of 15,723 documents are found in the SCOPUS database when searching for Microgrid OR Smart Grid OR Energy Systems Simulation, until the end of 2017. Most of these works have been published in the last decade, as can be seen by Figure 8.
As Figure 8 shows, although these topics have been briefly mentioned for more than 50 years, the interest in microgrid and smart grid simulation has just boosted in the last decade, following the large increase of investment in smart grids, and the worldwide definition of ambitious targets for renewable generation integration in the system, which requires the exploration of approaches such as the smart grid and microgrid, and their simulation. Table 10 shows the details on the most cited articles and most relevant authors in this domain.
As Table 10 shows, a significant number of documents related to microgrid and smart grid simulation can be easily found. A document discussing the requirements of smart grid simulation tools, including grid constraints, markets, legislation, communication infrastructure, regulation, and legislation, can be consulted in [190], paving the path for development ways of future simulators. Also, a relevant list of smart grid simulation tools can be consulted in [191]. In many of these works, a multiagent approach is commonly applied. Many of the advantages of using multiagent systems in the energy system are discussed in [124]. These advantages are mainly related to the large-scale accommodation of renewable energy sources. The distributed architecture is one of the emphasized advantages, due to the distributed nature of renewable energy sources and their need for local decisions and information. The flexibility is another core asset, since a multiagent system can easily accommodate new agents and exclude agents (both software and physical resources) without compromising the simulated system. This is closely related to the resiliency of the system, as a distributed approach usually has a better response to changes and failure in the network, thus helping to improve its stability and efficiency [192].
In [193], three main types of approaches for smart grid and microgrid simulation using multiagent systems are identified, namely, centralized, distributed, and hierarchical approached. The centralized approach refers to the collection of homogeneous agents that are managed by centralized control and management agents. These two types of agents are described in [194] as reactive and cognitive, respectively. Reactive agents only respond to requests with their standard actions, and cognitive agents incorporate enhanced management and/or control capabilities. Another relevant work considering a centralized approach for agent-based control of microgrids is [195].
In a distributed approach, each agent detains the knowledge about his share of the system, and they interact with each other in order to coordinate and manage the operations in a distributed way, as well as to cooperate and share services that respecifies to each agent. A distributed system for self-organization of generation sources, consumption, and storage is proposed in [196]. The simulation of a test microgrid with DG, consumers, and storage is used in [197] to demonstrate the advantages of a distributed control using multiagent systems.
An hierarchical model considers the authority of some agents over others [198]. This approach requires classifying each individual agent as being of a certain type, which facilitates the definition of agents’ roles and also improves the scalability and robustness of real-time operational control [199]. The work presented in [200] proposes a three-level agent-based hierarchical approach for distributed power flow control. A simulation model based on a two-level hierarchical approach is presented in [201] to enable distributed microgrid control. The upper level comprises the central controller, which coordinates and manages the information from the network perspective. The bottom layer includes microsource controllers and load controllers, which perform their control and management locally. Market participation is also included in [202]. Agents in the higher levels are responsible for the decision-making regarding the complex tasks, while the lower-level agents are accommodated to simpler tasks. Some other applications of hierarchal agent-based systems for smart grid and microgrid simulation are discussed in [203–209].
A decentralized approach for the management of storage devices is proposed in [210]. An agent-based simulation infrastructure is used to support negotiations in order to optimize the operation of storage units in a decentralized manner. Another market-driven simulation system for storage device management and operation is introduced in [211]. This problem is defined as a multiplayer game, and the Nash equilibrium is used to minimize the energy cost by reducing the peak demand. Energy transaction patterns are analyzed through agent-based simulation in [212]. The load scheduling is addressed using genetic algorithms and considering the consumer comfort. The integration of renewable based generation in the electricity market is explored through simulation in [213]. This work explores potential business models considering the alternative models of DR for buildings. The market-based control of imbalances using a multiagent simulator is addressed by [214]. This simulator includes the control of different units and combines DR and several distributed energy sources.
An agent-based simulation approach for microgrid control is presented in [215], focusing on the distribution grid control decisions. The interaction between multiple microgrids is simulated and studied in [216]. This simulation model enables studying the coordinated control of multiple integrated microgrids, as well as incorporating relevant aspects such as reserve regulation. The simulation model presented in [217] focuses on large-scale modeling of transportation assets in smart grids. Namely, the simulator addresses the demand management of EVs through the simulation of the power grid and transport.
A simulation system for different types of consumers and their behavior in a smart grid setting is presented in [218]. This simulation system enables consumer agents to define their actions at each time, deciding among use of local generation, reduction of their consumption, trade of energy with utilities or their neighbors, and use of their batteries.
An agent-based simulation system for local energy systems named energy cultures is presented in [219]. The multiagent model integrates several types of agents, which are represented by aggregators in the wholesale market. This simulator has been used in the CASCADE project to address different issues, all related to the future role of the aggregator in future power and energy systems [220].
A flexible large-scale agent-based simulation tool for smart grids is presented in [221]. This simulator, named GridLAB-D, is open-source and considers different smart grid resources such as DR, storage, EVs, and the retail market. The flexibility of the simulator enables it to be used to study multiple questions in the field, related to energy trading, flexibility, management, and operation of smart grids.
The Multiagent Smart Grid Platform (MASGriP) is a large-scale simulator of microgrids and smart grids [163] that combines agent-based simulation with real-time simulation and physical emulation. This simulator can be used as a stand-alone application, using the agent-based system to simulate different types of smart grid players and management, operation, and interaction models, or it can be used in a physical setting to control a microgrid in real time, combining real resources with simulated players and assets. This simulator is connected to the MASCEM electricity market simulator [160], which enables combining the smart grid operation with market simulation.
A smart grid simulator that uses the AnyLogic simulation environment is presented in [222]. This simulator models local energy communities considering buildings with self-generation and storage devices. Different configurations are allowed in order to enable the study of a multiplicity of issues in the smart grid domain. The simulator introduced in [223] uses the HLA architecture to simulate multiple environments, among which is the smart grid. The proposed architecture connects different ICT components and applies it to power systems modeling. Additionally, the proposed architecture is integrated with models in Simulink, Omnet++, and JADE to enhance the modeling of smart grid applications.
The large-scale simulation of smart grids using Modelica is addressed in [224]. This object-oriented equation-based modeling tool enables increasing the flexibility, efficiency, and scalability of simulation frameworks. This work suggests that using such an approach may prove to be a relevant step forward in the simulation of large-scale systems such as the smart grid. The work presented in [225], in turn, applies the Discrete Event System Specification (DEVS) formalism for modeling and simulation of the smart grid. The proposed approach considers several distinct smart grid resources and shows that such a model enables specifying components as well as developing event-driven simulations.
The work presented in [226] proposes a cosimulation framework of the smart grid including the modeling of the power and communication networks. In [227], the same cosimulation subject is addressed, but this time focused on enabling direct load control in the smart grid. Issues such as the data volume, timing questions, and reliability issues are addressed. The PSS SINCAL and OPNET simulators are combined to enable the real application of the proposed simulation model in a German distribution network.
The large number of works related to microgrid and smart grid simulation leads to a rather vast and complementary simulation focus in this field. Table 11 presents a comparison of the main top-level characteristics of some of the most relevant simulators discussed in this paper.
As can be seen from Table 11, the main focus of microgrid and smart grid simulators is the study of the integration of renewable energy generation in the system. Grid modeling is almost always available, and DR is also being increasingly addressed by most recent works. On the other hand, the simulation study of EVs, the role of the aggregator, and the market interaction with the local grid are issues that, although addressed by some simulators, still lack a significant wide spread in terms of simulation modeling. The modeling and simulation of agent learning capabilities is still very poor in this domain, contrarily to what can be seen from the field of electricity market simulation. It is a fact that most of the least explored areas that are discussed here are addressed in dedicated works, which does not mean these areas are not being addressed. However, their integration in complete simulation frameworks is still showing to be taking its first steps.
6. Final Remarks and Future Research Directions
This survey analyzed recent research (mostly from 2010) on complex optimization and in the new paradigm of power systems and the smart grid era. The main focus is centered on (i) the optimization of power systems under the new smart grid context, (ii) the discussion and analysis of the application of computational intelligence approaches as means to solve the complex optimization problems, and (iii) the analysis of large-scale simulation of electricity markets and smart grid.
It started with a preliminary section with a brief explanation of basic concepts in the new power systems paradigm. After that is an overview of research work to tackle challenges brought by the smart grid technologies. Then, it focused on research related to energy resource management due to its particularly complex case in this field. This study realized that smart grid optimization is starting to adopt methods to tackle sources of uncertainty in the mathematical models. However, it is found that most of the work relies on traditional approaches and lacks ability to scale to real systems (due to limited computer resources with the adopted approaches). As a consequence, they are also limited in covering all sources of uncertainty and considering different problem horizons.
The survey also analyzed CI due to its large potential to tackle the challenges introduced early in this survey related to smart grid optimization. In fact, CI encompasses some of the most existing paradigms of computer sciences. Among them, EC is one of the most prominent paradigms, with some very mature algorithms applied to a wide variety of complex optimization problems. In the energy domain, hybrid techniques that make use of the three main paradigms of CI (i.e., EC, ANN, and FS) are currently used as a tool to model and solve some of the arising issues in power systems. The evolution of complex energy systems is pushing into the limits on the application of the most accepted exact solutions, which in some cases are not suitable to deal with these new complex scenarios. It is in this context that motivates the use of CI as an efficient tool to deal with this challenging scenario. However, despite the successful applications of CI in a wide variety of problems, the portfolio of algorithms belonging to CI has the drawback of being unable of guaranteeing optimal solutions. The use of additional resources, such as supercomputers or parallelism, open a window to refine the application of CI, and it is expected that in the near future, CI solutions will be sufficiently mature to be adopted as the preferred solutions to power systems.
Due to its importance to the new paradigm of power systems, we analyzed electricity market simulators, which together with smart grid optimization are also a complex case of power systems studies. We realized in this study that those simulators are valuable but still limited solutions, as they are usually directed to specific market environments and present limitations in coping with the interaction with external systems. This hardens the possibility for cosimulation of distinct and complementary environments, such as the coexistence and simultaneous participation of players in multiple electricity markets. Most of the existing simulators include some agent learning capabilities, although these are almost entirely exclusively directed to generation agents. Many of these simulators also consider grid constraints, making them suitable for simulation under operators’ and regulators’ perspective. However, the study of the integration of distributed renewable generation, integration of DR, of electric vehicles, or even of aggregators in the market, is still very poor. This deficiency makes most of these simulators still unable to provide a valuable support for the study of the evolution of the power system in accommodating the targeted large amounts of renewable generation and to incentivize the active participation from the consumers’ side.
Effective solutions should rapidly appear, so that current and alternative market models can be easily simulated and assessed using realistic models regarding not only electricity markets but also the present and future penetration of diverse types of energy resources. The main problems of the field such as the massive integration of renewable energy sources in electricity markets, the inadequate models to support the active participation of consumers, and the consequent need for modeling alternative and innovative market models that enable accommodating new types of players, remain, therefore, unaddressed.
On the other hand, the quick emergence of the process of smart grids is showing not to be entirely free of problems. A large number of practical applications, although very expensive, are enabling solutions that present serious limitations and provide little return of investment. It is not clear that the rolled-out equipment is sufficiently open and flexible to be useful for the next generation of smart grid solutions that should appear in the coming years. Additionally, although important contributions are being achieved, these still remain as solutions for partial problems. In highly dynamic and codependent areas, such as power networks, smart grids, and electricity markets, the cooperation between different systems becomes essential in order to look at the global problem as a whole. Most of the smart grid-related works consist in practical implementations, highly industry driven, and involving almost exclusively large stakeholders in the field, such as regulators, operators, and utilities, resulting in an almost complete focus on achieving fast ways to overcome present problems. This is, however, leading to an absence of pure research exploitation in the field, which is essential to reaching innovative findings to enable the scientific breakthrough that is required to provide advanced solutions, not only to the current and most prominent problems that are arising but also to future issues that are inevitable.
The focus of current microgrid and smart grid simulators can be said to be quite diversified. The study of the integration of renewable energy generation in the system is the main addressed question, while grid modeling is usually addressed, and DR begins to be considered in large scale. On the other hand, relevant issues such as the simulation of electric vehicles, aggregator models, and the interaction with the market are still not sufficiently addressed by current large-scale simulators. Additionally, most simulation approaches are based on multiagent systems. Although this paradigm is showing to be suitable for the task, it still presents several limitations, as identified in [193]. The emergent or unexpected behavior from autonomous agents in dynamic environments is one of the main sources of uncertainty, as the same level of behavior cannot be guaranteed under different settings, scenarios, and contexts. The portability of agents is another issue, which relates to agents’ physical integration. Although many advances have been accomplished in recent years, issues with standardization and difficulties of communication with different types of devices and resources are still proving to be an arduous challenge to overcome. The scalability is another important aspect in large-scale simulation. There are multiple approaches devoted to guaranteeing the scalability of the simulation environment; however, these are still always dependent on the processing machines when it comes to increasing the number of agents and integrated smart grids or microgrids. Finally, the security component is another important limitation when considering the real application of agent-based systems or their simulation using real data. Security in communications, data storage, and management still need to be highly improved and standardized so as to enable the wide-scale application of agent-based simulators in real environments.
Appendix
Statistic of recent research in the scope of energy resource management
Additional information is given in this appendix regarding statistics of recent research in scope of energy resource management. Figure 8 shows the number of documents produced between 2010 and 2016 by affiliation country (top 12) for the search term “energy resource management” OR “energy management system” in the SCOPUS database. China-affiliated researchers lead the number of documents produced so far with 674 documents, which combined with United States, Japan, and Germany represent more than 50% of the documents produced in this research field. Portugal appears in the 12th position of this list (6th European country) with 132 research documents.
For the same search term, we also investigated the research impact of the documents produced in this field, that is, number of citations and citations per paper, which can be interpreted as quality indicator. Table 12 shows the total number of citations by country (the same as appears in Figure 9), and Table 13 shows the citations per paper for the same countries. It can be seen that United States, China, and Canada are the research affiliations with the highest impact translated by the total number of citations between 2010 and 2016 (51% of top 12 global citations). It is interesting to remark that in spite of Japan ranking third in the total number of documents (312, see Figure 9), the total number of citations of those documents is only 488, which puts Japan as the last in this list regarding the total number of citations, with 1.6 citations per paper (see Table 13). Data suggests that Canada and Spain perform quite well in the research impact for the advances in the energy resource management, figuring with 14.8 and 10.5 citations per paper, respectively, which are significantly apart from the other countries in this list. Portugal also presents very interesting impact results ranking 5th in the citations per paper list, in spite of being in the 12th position when ranked by number of documents (see Table 13). Portugal is ahead of United Kingdom and Germany, India, and Japan when seen by for the total citations (see Table 12). This data is accurate on how research affiliations have performed in the past, namely, between 2010 and 2016, and according to SCOPUS database. But we understand that highly cited articles related to the search term of energy resource management, such as [11] (Canada), [33] (Canada and USA), and [1] (Italy), contribute to improve the country affiliation rank.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The present work has been developed in part under the EUREKA-ITEA2 Project M2MGrids (ITEA-13011) and Project SIMOCE (ANI|P2020 17690) and has received funding from FEDER funds through COMPETE program and from national funds through Fundação para a Ciência e a Tecnologia under the project UID/EEA/00760/2013. In addition, it was supported in part by Grant agreement no. 703689 (project ADAPT). Additionally, this work has profited from ideas from the proposal of the recently funded project with ref. 02/SAICT/2017/28983 from CENERGETIC. This work has been also partially supported by Consejo Nacional de Ciencia y Tecnologia (CONACYT), Mexico.