Abstract
We consider algorithmic design, enhancement, and improvement of evolutionary computation as a mechanism design problem. All individuals or several groups of individuals can be considered as self-interested agents. The individuals in evolutionary computation can manipulate parameter settings and operations by satisfying their own preferences, which are defined by an evolutionary computation algorithm designer, rather than by following a fixed algorithm rule. Evolutionary computation algorithm designers or self-adaptive methods should construct proper rules and mechanisms for all agents (individuals) to conduct their evolution behaviour correctly in order to definitely achieve the desired and preset objective(s). As a case study, we propose a formal framework on parameter setting, strategy selection, and algorithmic design of evolutionary computation by considering the Nash strategy equilibrium of a mechanism design in the search process. The evaluation results present the efficiency of the framework. This primary principle can be implemented in any evolutionary computation algorithm that needs to consider strategy selection issues in its optimization process. The final objective of our work is to solve evolutionary computation design as an algorithmic mechanism design problem and establish its fundamental aspect by taking this perspective. This paper is the first step towards achieving this objective by implementing a strategy equilibrium solution (such as Nash equilibrium) in evolutionary computation algorithm.
1. Introduction
Game theory is the methodology used to research strategic interaction among several self-interested agents [1]. Some important concepts, such as type, strategy, and utility, are useful to an understanding of the theoretical framework of game theory. Agent type indicates the preferences of the agent over different outcomes in a game. A strategy is a plan or a rule, which defines the actions that an agent will select in a game. The utility of an agent determines different allocations and payments under its and other agents’ types and strategy profiles; for example, an agent rationality in game theory is to implement the expected utility to be maximum. An agent will select a strategy that maximizes its expected utility, given its preferences with regard to outcomes, beliefs about the strategies of other agents, and structure of the game.
Nash equilibrium (NE) is one of the solution concepts that game theory provides to compute outcomes of a game from self-interested agents under certain assumed information that is available to each agent, such as agents types and strategies [2]. It states that every agent in a game should select a maximum utility strategy taking account of other agents’ strategies so as to achieve equilibrium. The fundamental aspect of game theory lies in the Nash solution concept which, however, requests stronger assumptions on agents’ information (type, strategy, utility, etc.). There are some related solution concepts, such as dominant strategy and Bayesian-Nash strategy in game theory. In this paper, we initially attempt to implement Nash solution concept in evolutionary computation (EC) in the context of a mechanism design problem.
The history of mechanism design can be traced back to the 1920s to the 1930s, when there was an economic controversy concerning the socialist economic system. Liberal economists, such as von Mises and von Hayek, believed that socialist economics cannot obtain effective information to make their economic system operate efficiently [3, 4]. However, other economists, such as Lange, thought socialist economics can solve the problem of requesting more information about economic operation so as to promote efficient resource allocation [5]. The contention of this controversy focuses on the information issue, which is also a core problem of mechanism design. Hurwicz established the fundamentals of mechanism design theory from the information viewpoint, which proposed a common framework to compare the issue of efficiency among different economic systems [6].
Mechanism design theory can be considered as a comprehensive utilization of game theory and social choice theory, which is referred to as principal agent theory and implementation theory as well. Its primary philosophy is to design a series of rules to implement the trust between principal and agent and to ensure the mechanism runs well under an asymmetric information condition. The fundamental issues of mechanism design refer to (1) whether there is a set of rules in a game and (2) how to implement these rules. The objective of mechanism design is to achieve a preset objective that a game establishes, when all agents act for their own benefit with their private information.
When we make reference to individuals in EC, they are neither rational nor self-interested participants in an EC algorithm. The individuals just follow the fixed rules of the EC algorithm. In this paper, we assume that they are rational, self-interested, and more “intelligent” agents in an EC algorithm, so that we can handle the design of EC algorithm as a mechanism design problem. The design principle of an EC algorithm should be more reasonable based on this assumption, rather than merely simulating natural phenomenon in an iterative process of evolution. There are several issues that need to be discussed in this context: (1) what kind of information should be involved in mechanism design of EC algorithm; (2) how to distinguish principal(s) and agent(s) in a game of EC algorithm design; (3) which solution concept should be implemented in this game; (4) how to define the expected utility function for an established mechanism or model; (5) whether the established mechanism or model is the optimal one; and (6) what are the characteristics and properties of designed game induced by EC. After modelling EC as a game, we can introduce theoretical principles of game theory and microeconomic theory into the fundamentals of EC for designing, enhancing, and improving more efficient and effective EC algorithm. This is the primary motivation and original contribution of this work.
Following this introductory section, in Section 2, we make a brief review of some basic definitions, notations, and related works on game theory, mechanism design, and EC. The primary concepts and works include mechanism design, Nash equilibrium, EC, and interdisciplinarity of these aspects. We establish some basic concepts on algorithmic mechanism design of the EC and introduce a formal framework that implements Nash equilibrium into EC for solving strategy selection issue in Section 3. The formal framework can be implemented in any EC algorithm by designing agent, utility, strategy, and equilibrium solution (such as Nash equilibrium, Baysian-Nash equilibrium, and dominant equilibrium) in corresponding EC algorithm in detail. In Section 4, we design the proposal in a differential evolution algorithm and evaluate its performance by designing a series of comparison experiments. Based on the evaluation results, we make a comprehensive discussion on algorithmic mechanism design of the EC, Nash equilibrium implementation in EC, and so forth, in Section 5. Finally, we present some future works and open topics towards the final objective of our proposal, that is, an algorithmic mechanism design of EC, in Section 6.
2. Definitions, Notations, and Related Works
2.1. Mechanism Design Problem
The mechanism design problem can be described as follows. There is a system with agents () and a set of outcomes (denoted as ). Each agent has private information about its utility corresponding to the outcome. Other information in this system is public knowledge or public information. The private information is the agent’s type (denoted as ) that determines its utility () among different outcomes (). The set of type () presents all the possible information available to the agent . The mechanism design problem is to solve an optimal solution (, )) of an implementation problem given by (1) with agents that have private information about their actions. Equation (1) is a social choice function, which generally selects an outcome with maximum total utility in the system (2) [8]. Consider
Mechanism presents a set of strategies () of each agent and methods used to select an outcome based on the strategies. Game theory is used for analysing the outcome of a mechanism. On the contrary, the mechanism design is used for implementing a social choice function (1), if the outcome can be obtained by an (some) equilibrium strategy(ies). Finding an equilibrium strategy is the final objective of mechanism design. The equilibrium strategy can be Nash equilibrium, Bayesian-Nash equilibrium, dominant equilibrium, or other solution concepts. The better solution of a mechanism design is as strong as possible.
2.2. Nash Equilibrium
Nash equilibrium (NE) is a well-known solution concept in game theory, which computes the outcome of a game with self-interested agents based on certain assumptions on information available about each other [2]. It states that every agent selects a strategy with maximizing utility in equilibrium by considering every other agent’s strategies. Suppose that is a joint strategy of all the agents (there are agents) and is a joint strategy of all the agents except agent . A Nash equilibrium is a strategy profile , if every agent maximizes its utility. It is shown in (3), where . Note that and present the strategy of agent under its type and strategies of other agents without agent under their types , respectively:
A strategy profile is in Nash equilibrium that means every agent can obtain maximum utility with strategy . The disadvantage of Nash equilibrium is that it makes very strong assumptions about agent types and information about other agents. It must have relative enough information about preferences of every agent and agents with common knowledge to play a Nash equilibrium in a one-shot game. There is as well an important issue that all agents must select the same Nash equilibrium.
2.3. Evolutionary Computation
Evolutionary computation (EC) is a series of stochastic optimization algorithms that are inspired originally by natural selection and “survival of the fittest” and further developed by ant colony optimization, artificial immune systems, partial swarm intelligence, and others. From the algorithmic taxonomy viewpoint, EC takes the probability theory as its philosophy and methodology. The fundamentals of its search mechanism are established in the basics of probability theory. The system behaviour of an EC algorithm can therefore be presented as a probability transition matrix, and its dynamic optimization process can be described as a Markov chain [9]. However, other theoretical analysis methods from deterministic theory, such as fixed point theory, are also introduced into the EC in order to study its fundamental aspects [10], such as efficiency, effectiveness, and convergence. Drawing inspiration from chaos theory and its ergodicity, a chaotic evolution algorithm has been recently proposed and studied [11]. This is very different from the conventional deterministic and stochastic optimization algorithms. Chaotic evolution can be considered as an implementation of the chaotic optimization algorithm, whose theoretical fundamental is supported by chaotic philosophy and methodology.
There are two components in EC from the viewpoint of the algorithm design framework. One is an iterative process, and the other is one or several evolutionary operations that are implemented by a variety of methods. Algorithm selection and parameter settings are two critical issues, when we apply EC to an optimization problem. The objective of the former issue is to answer the question as to which is the best EC algorithm to solve a concrete problem. The latter one seeks to obtain the best parameter setting of an EC algorithm to obtain a better optimization performance. This paper attempts to solve and answer the latter problem from the viewpoint of algorithmic mechanism design and tries to find the fundamental aspects of EC from this perspective.
2.4. Differential Evolution
Differential evolution (DE) is a population-based optimization algorithm, as well as a member of an EC family [12]. DE uses a differential vector from two random individuals to perturb a base vector (a third random vector from a population) to implement a mutation operation and obtain a mutant vector. It conducts a crossover operation between the mutant vector and proceeded target vector to create the trial vector. Following this, it compares the fitness of the target vector and trial vector to allow the better one to survive into the next generation. The formal expression of this search mechanism is shown in (4), where is a mutant vector, is a base vector, and and are two random vectors ( and are the indexes of individual and dimension, resp.). is a scale factor that needs to be set. Note that the target vector, base vector, and two random vectors are four different vectors, so the minimum population size is four in DE. The pseudocode of DE is shown in Algorithm 1. Consider
| ||||||||||||||||||||||||||||||||||||||||||
For improving DE performance, a self-adapting control parameter method is proposed in DE (jDE) to implement a variety of selections of DE parameters [7]. The crossover rate () and scale factor () are encoded in each individual in jDE, which are adjusted by (5) and (6) every generation, respectively. The parameters, and , are both set to , and is a random value from a uniform distribution within . From (5) and (6), and are updated in a small probability:
2.5. Evolutionary Computation Meets Game Theory and Mechanism Design
Game theory attempts to determine the outcome of a game with a set of given strategies from self-interested agents, and mechanism design seeks to design the strategies of agents to obtain the desired outcome in a game. The research objectives of game theory and mechanism design are to find outcomes corresponding to strategies and to design strategies under a desired outcome, respectively. However, EC tries to find the optimal solution(s) of an optimization problem. These three disciplines, game theory, mechanism design, and EC, are quite different with regard to research philosophies, approaches, or objectives.
Game theory and mechanism design have been introduced into some fields, such as distributed artificial intelligence [13], resource allocation [14], and scheduling [15]. It is easy to establish concrete models of agent and utility function by using game theory in corresponding applications. Mechanism design was also reported in computation related topics [16] and algorithmic design problems [17], which presents complexity bounds and worst case approximation. These studies focus definitely on the same viewpoint, that is, a mechanism design problem of deterministic optimization. These works do not relate to EC and do not pursue our proposal, that is, an algorithmic mechanism design of the EC, which is a stochastic optimization method rather than deterministic optimization one.
From the related literatures and to the best of our knowledge, EC can act as an optimization tool to find the best response, equilibrium of strategy in game theory and mechanism design. Some EC algorithms, such as genetic algorithm [18], genetic programming [19], and coevolution [20], are applied to game theory problems to obtain the best strategy or parameters. Scant literature reports having applied the philosophy and methodology of game theory or mechanism design to the fundamental aspects of EC. This paper attempts to conduct some initial work in this area.
3. Algorithmic Mechanism Design of Evolutionary Computation: A Strategy Equilibrium Implementation Problem
3.1. Motivation of the Proposal
Conventional EC algorithm as a search method is applied to an optimization problem with a set of fixed algorithm parameters and operations. Although the inspiration of EC seeks to find adaptive mechanisms in its search scheme, the fixed parameter setting restricts its optimization capability. From the system theory viewpoint, the whole EC algorithm system can be considered as a control system and its parameters decide the system behaviour. If we aim to obtain the best optimization performance, the parameters of EC algorithm should be controllable, and the relationship between parameter settings and optimization performance should be observable. However, because the EC algorithm belongs to stochastic method, such deterministic methods (e.g., automatic control method) have not been applied to the EC area in order to study on its theoretical fundamentals.
There are primary three research directions for improving optimization performance of an EC algorithm [21]. The first is obtaining information from a fitness landscape, such as fitness landscape approximation, and using the information to conduct a special operation or to develop new search schemes for tuning the parameter of an EC algorithm [22]. The second is developing new mechanisms in extant EC algorithm to enhance its performance or to implement the parameter adaptive mechanism [23]. The jDE algorithm that is introduced in Section 2.4 belongs to this aspect. The third direction is creating new EC algorithms or metaheuristics with less parameters and more natural adaptiveness in order to enhance EC algorithm [11].
Individuals in conventional EC algorithms or in some population-based optimization algorithm are common elements, which present the search space and structure aspect of an optimized problem. Individuals search for the optimum/optima with the information shared between one another, so they are influenced each by the other from one generation to the next. They operate under the same evolutionary operations with fixed operation rates from parameter settings in a certain EC algorithm. This work scheme restricts the EC algorithm search capability.
If we consider individuals in EC algorithm as agents, the EC algorithm therefore can be considered as a game, whose outcome is optimal solution(s) or some other metric(s). Furthermore, these agents play the game (i.e., EC algorithm) with self-interested preference and conduct optimization strategies with their own preferences. An EC algorithm designer can play this game by using noncooperative or cooperative game concepts. The objective of the EC algorithm design is to find optimum/optima by designing a proper strategy for these individuals (i.e., agents). This description can be abstracted as a mechanism design problem. That is, design, parameter setting, and operation selection of the EC algorithm can be decided by the individuals, and the desired outcome is to find the final optimum/optima. EC algorithm can be modelled as a game, that is, an agent system, so the corresponding theoretical fundamentals of agent system, game theory, or mechanism design can be brought to bear on the study of the fundamental theoretical aspects of EC algorithm. This philosophy and motivation highlight the originality of this paper.
3.2. Strategy Equilibrium Implementation in Evolutionary Computation Algorithm
We propose that the design of an EC algorithm can be considered as a mechanism design problem. In this section, we briefly introduce some concepts in game theory and their corresponding explanations and implementations in EC. Based on these concepts, we establish a formal EC algorithm framework by using the equilibrium concept and solve this mechanism design problem by finding the Nash strategy equilibrium in EC algorithm.
3.2.1. Agent and Its Type
An agent is an abstract concept in game theory. It refers to a participant in a game who will make strategic decisions based on its type. Type () of an agent determines preference of the agent by considering all the outcomes of a game, as it is mentioned in (1), (2), and (3). As usual, refers to the type of an agent in game theory.
Individuals in EC are definitely considered as agents under the basic philosophy of our proposal. In game theory, an agent is treated as being self-interested. However, in EC, we consider it as a more rational one that allows itself to select certain operations, even though its utility will become low. For example, the simulated annealing mechanism in EC is such a case, if the EC algorithm allows individuals to be replaced by their offspring with worse fitness. The agent in a game can determine its own behaviour, so the individuals of EC should follow this rule by encoding operation types and their rates in themselves. In the EC, the type of an agent can be considered as information, such as fitness and fitness landscape, or some metrics of the evolution.
3.2.2. Strategy
Strategy concept in game theory presents a set of actions, or decision rules, which define the action of an agent. It is decided by the agent type (), and presents agent selects strategy due to its type . The strategy of an agent can be deterministic or stochastic. In game theory, the deterministic strategy and stochastic strategy are referred to as pure strategy and mixed strategy, respectively. A strategy is a function of the agent type, as well as being an expression of the utility of the agent. They are interdependent concepts, whereby the utility of an agent is a function of the strategy. EC operators and parameters and their selection issues can be considered as the strategy and strategy selection problem in EC algorithm design. They can be decided by each individual according to their own type, that is, the information obtained in the evolutionary search process.
3.2.3. Utility
The utility of an agent is a function that decides preference of agent under all the outcomes () and types (). An agent prefers outcome to outcome , when (note that and is a parameter), so the measurement of utility is used to decide agent strategy. The utility concept in EC refers to some measurements of algorithm optimization performance, such as fitness improvement, evolution successful rate, or other metrics that can evaluate algorithm performance or individual performance.
When we describe a mechanism design problem, two issues should be stated clearly. One is the domain of agent preference that determines what an agent refers to, and the other is solution concept, that is, what the outcome specification is. In this paper, we primarily discuss the second issue, that is, implementation of the solution concept of an EC algorithmic mechanism design, and simplify that the agent (individual) has a quasilinear preference. In (7), is valuation function, is a choice of agent, and is payment by agent. In our evaluation section, we use fitness improvement (8) as the utility (it presents valuation and payment of an agent as well) to determine an individual’s strategy:
3.2.4. Strategy Equilibrium
The agent in EC algorithm is the individual, which is a participant in a game of EC. Each individual can decide their own strategy, that is, operation and operation rate, by their utilities and types, which can be measured as fitness improvement information or other algorithm evaluation metrics. All the EC algorithm implementations can be abstracted as a mechanism design problem, whose equilibrium concept is a solution of the problem. As all types of equilibrium concept can be a solution of our established framework, a Nash equilibrium is a related easy and weak solution in a game because of . The objective of mechanism design is to implement some strategy equilibrium concepts in a game; however, it is an optimization problem, in which equilibrium implementation can lead to the best performance and fast convergence of EC algorithms. We initially discuss, design, and evaluate our proposed framework by implementing Nash equilibrium, but it is not limited to within Nash equilibrium.
3.2.5. Mechanism Design of Evolutionary Computation Algorithm
We define and describe EC algorithm as a mechanism implementation problem. Given an EC algorithm , it defines a strategy set to each individual and an outcome rule that is the outcome implemented by the mechanism (EC algorithm) for strategy profile . EC algorithm (a mechanism, ) implements social choice function (1), if and . Strategy is an equilibrium solution to the game induced by EC.
There are some different aspects in this description of EC, when we consider it as a mechanism implementation problem. First, individuals of EC algorithm can select their own strategy depending on their types, that is, the information they obtain, rather than process a fixed EC operation in conventional EC. Second, we should define type, strategy, and utility of individual to describe the game induced by EC. Third, we should compute some forms of equilibrium solution for an EC algorithm implementation in every generation or several generations () once for achieving best optimization performance of EC. Fourth, the domain of agent preference should be defined reasonably to achieve the correct decision-making of strategy selection. From this description of EC by considering it as an algorithm mechanism design problem, EC algorithm is a set of algorithm strategies and outcome rules. The optimization performance of EC algorithm is decided by strategy and strategy selection. The study of EC convergence or performance improvement is transferred to the study of the relationship between strategy selection (equilibrium implementation) and EC algorithm performance. It provides us with an opportunity to discuss the theoretical aspects of EC within this framework.
3.3. A Case Study: Nash Strategy Equilibrium-Based Differential Evolution Algorithm
After we introduce the optimization process of EC as an algorithmic mechanism design problem, there are a variety of ways to implement EC algorithms by designing concrete implementations of a game. In this section, we design a concrete implementation of an EC algorithm by considering Nash strategy equilibrium and take DE algorithm as a case to study how to design our proposal in an EC algorithm. There are two design issues that should be concentrated on especially. One is the definition of strategy; the other is equilibrium calculation. In Nash strategy equilibrium-based DE, operations and their parameters are coded with each individual, so that the individual can select their own operation and rate. Mutation method, crossover rate, and scale factor rate are three primary parameter settings in DE. For simplifying the design objectives, we design the algorithm by splitting individuals into two groups (Group A and Group B) with equal population size and strategy sets within the following criteria:(i)Mutation = .(ii)Crossover rate () = .(iii)Scale factor () = .
There are two participants (Group A and Group B) who join in the designed DE algorithm. Every individual in the designed DE algorithm can select a mutation method from either “DE/best/1/bin” or “DE/rand/1/bin,” crossover rate, and scale factor either more than 0.5 or less than 0.5. We define the utility (payoff) of agent (individual group) as the fitness improvement (8), where and are fitness of individual in and generations. For example, for a minimum optimization problem, the less a fitness improvement value the better, but our proposed framework is not limited to within this definition of utility. After one generation or several generations ( in Algorithm 2), we calculate the fitness improvement of every individual and sum total fitness improvements of Group A and Group B with different strategies and obtain the payoff matrix of different strategy selection combinations (Table 1). We have
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
With the payoff matrix and the definition of Nash equilibrium as in (3), we can find that the strategy , which satisfies (), is a Nash strategy equilibrium in our designed DE. From this definition, we can find that is a dominant solution of the game as well. In our established EC framework, a Nash equilibrium solution can be a Pareto solution and dominant solution. If Nash equilibrium solution is a Pareto solution, it presents maximal optimization capability of the algorithm that decides the convergence property of the algorithm. This is a critical property of our proposal. In the next generation, if the individual strategy is the Nash equilibrium strategy, they keep their strategies as they are. Otherwise, an individual will tune its strategy with a random guess. By considering these design elements into a canonical DE, Nash strategy equilibrium-based DE algorithm is shown in Algorithm 2.
4. Numerical Evaluations
4.1. Evaluation Design
The objective of our proposal is to design an EC algorithm from the viewpoint of game theory and mechanism design. After we find the Nash equilibrium strategy, the EC algorithm should be enhanced practically. In the fundamental aspects of EC algorithm, approaches and theories of EC will be therefore explained by the corresponding context of game theory and mechanism design. In this section, we design a series of evaluations to verify the effectiveness and efficiency of our proposal.
We use 14 benchmark functions from [24] in evaluation. Table 2 shows the type, characteristic, bound, and optimum of these benchmark tasks. They include landscape characteristics of being shifted, rotated, global on bounds, unimodal, and multimodal. We test them with three types of dimension setting, (5 dimensions (5D)), (10D), and (30D).
4.2. Evaluation Setting
We compare our proposed Nash strategy equilibrium- (NE-) based DE algorithm (i.e., best-, rand-, best-Cr, rand-Cr, mutation, and mixture) with canonical DE and jDE. Table 3 presents the abbreviations of algorithms and their parameter settings, scale factor (), and crossover rate () of two canonical DE (DE-best and DE-rand) are both set at 1 from the discussion of [12]. We test each function with 30 independent runs and 1000 generations for each run. Every benchmark function is a minimum optimization problem, that is, the smaller fitness value means the better optimization performance.
Because the production of differential vectors in DE is a critical factor that influences optimization performance and one of our evaluation objectives is to investigate the strategy selection influence of the DE algorithm, we need to reduce the diversity of differential vectors in the evolution so as to reduce population diversity influence and to enhance strategy selection influence. As the minimum number of a DE population size is four (), we therefore use it in our evaluation to reduce the diversity influence as much as possible. For a fair comparison of these algorithms with normal conditions, we also set the population at to check their performance from discussion of [12]; that is, , , and for , , and , respectively.
In NE-based DE algorithm, there is a parameter ( in Algorithm 2) to decide in which generation an NE strategy is calculated. In our evaluation, we set that means the NE strategy is calculated every generation. If individuals have these NE strategies, they keep them; otherwise, they select a strategy from a strategy pool with a random guess.
4.3. Evaluation Metrics
Several evaluation metrics are involved in the analysis and discussion. We apply Wilcoxon signed-rank test and Friedman test on the fitness value at 1000th generation to check the significance of the proposal and make an algorithm rank. Tables 4, 5, and 6 are the mean values of each benchmark function with 5D, 10D, and 30D, respectively. Wilcoxon signed-rank tests are applied between comparison algorithm and our proposed NE-based DE algorithms.
After obtaining the average mean rank of each algorithm from Friedman tests, we apply Bonferroni-Dunn tests in significant level of , , and on 5D, 10D, and 30D benchmark problem. Critical difference (CD) used in Bonferroni-Dunn test is in (9), and and , is equal to , , and from Appendix Table B.16 of [25]. Figure 1 demonstrates the visual presentations of critical differences among these algorithm ranks. We have
Number of function calls ( in (11)) before the desired reached fitness value ( in (10)), accelerate rate ( in (13)), and success rate ( in (12)) are introduced as well to evaluate our proposal. Tables 7 and 8 present these results. Consider
5. Analyses and Discussions
5.1. Philosophy of the Proposal
A good researcher should think outside the conventional philosophy and methodology of a particular field and consider a broad view rather than focus on small matters. The objective of this study is not only to find a method for designing, enhancing, and accelerating EC from the viewpoint of an algorithmic mechanism design problem, but also to establish EC fundamental aspects by borrowing from game theory and mechanism design.
There are three parallel ways to research and consider our world from the philosophies of determinism, probability, and chaos. In the optimization field, there are also three categories of optimization method from the corresponding philosophy and methodology, that is, deterministic, stochastic, and chaotic optimization methods. EC belongs to the stochastic one, and its fundamental aspect should be described from the probability viewpoint. This restricts the fundamental development of EC and explanation capability of its algorithms. This study tries to use fundamentals from game theory and mechanism design (deterministic theory) to explain EC (stochastic algorithm) and establish its fundamental contents. This is one of the contributions of this paper.
From the description of EC in Section 3.2.5, EC is a mechanism that is composed of strategy and outcome rules. Different equilibrium decides the optimization performance of an EC algorithm. EC convergence lies in the effective strategy equilibrium design and implementation. This is another contribution made by this paper. This provides another view for the understanding of EC algorithms from the fundamentals of game theory and algorithmic mechanism design.
5.2. Performance of the Proposal
From the observations of mean values in Tables 4, 5, and 6, our proposal can obtain a significantly accelerated convergence in benchmark functions. In Table 4, most of the proposals outperform DE-best and DE-rand, and some of them outperform jDE-best and jDE-rand. Optimization performance applied to multimodal tasks by means of our proposals is better than that applied to unimodal tasks. The same observations can also be made with regard to the 10D and 30D benchmark tasks in Tables 5 and 6. This may be because the capability of escaping the local optimum by means of our proposal is better than that of DE and jDE. However, the deep exploitation capability of our proposal is weak. This indicates that the strategy selection of our proposed EC algorithm has both high exploration capability and weak exploitation capability. It needs to be further investigated and proved by both empirical and theoretical studies in our future work.
Algorithms with NE strategy scale factor are more effectively applied to multimodal 5D benchmark tasks than to unimodal ones by comparing best- and rand- algorithms applied to the group and group in Table 4. The same conclusion cannot be found in 10D and 30D benchmark tasks in Tables 5 and 6. It seems that the population diversity caused by scale factor is more efficient for the improvement of optimization performance in high dimension tasks than that in low dimension tasks. Algorithms with NE strategy scale factor have lower acceleration performance than those with NE strategy crossover. This can be explained from the viewpoint of population production diversity by these two mechanism implementations.
Algorithms with NE strategy crossover are more effective in all dimension settings, that is, 5D, 10D, and 30D benchmark functions. When the complexity of a task increases, the effectiveness appears more explicitly. This can be observed by comparing Wilcoxon signed-rank test results between the two algorithms (best-Cr and rand-Cr) in 5D, 10D, and 30D benchmark tasks in Tables 4, 5, and 6, respectively. The effectiveness and efficiency of our proposal in high dimension and complex benchmark problems are demonstrated in this way.
In Tables 4, 5, and 6, only a few benchmark tasks are accelerated by our proposal with NE strategy mutation, where an individual selects either the best vector or a random vector as the best vector. This indicates that individual searching in the different region (around the best individual or a random individual) is not a serious factor of performance influence in DE with our proposed NE strategy.
5.3. Nash Equilibrium Implementation Inclusion
Mixture algorithms include three NE strategies, that is, NE strategy of , NE strategy of Cr, and NE strategy of mutation. It can obtain the best optimization performance from these algorithms with a single NE strategy because of implementation inclusion. The evaluation results and Wilcoxon signed-rank tests from Tables 4, 5, and 6 do prove this conclusion. However, the performance of algorithms with mixture NE strategies is not completely equal to algorithms with single NE strategies. For example, best-Cr and rand-Cr are two algorithms with more winner cases from Wilcoxon signed-rank test, but this is not the same as the winner cases of mixture algorithms in Tables 4, 5, and 6.
From the NFC, SR, and AR metrics (Tables 7, 8, and 9), rand-Cr algorithm is the one with the least NFC and greatest SR, and jDE-best and mixture algorithms have the first and second winners in AR metrics in all dimensional setting benchmark tasks, respectively. These observations also indicate that DE with our proposed NE strategy can achieve the desired fitness improvement target with the less computational cost. Although the mixture algorithm is not the winner, there are not many differences between AR values of jDE-best and mixture in all dimensional setting benchmark functions. From a statistical viewpoint, they are not significantly different. EC with pure strategy (canonical DE) has less optimization performance than that obtained by EC with mixed strategy (proposals) from our empirical evaluation. Although some other researches have the same empirical evaluation result, they do not all explain the reason for EC algorithm improvement from the viewpoint of game and mechanism design.
5.4. Algorithm Rank
We apply Friedman test on the 10 algorithms in our evaluation. The algorithms, best-Cr and mixture, win the most of first ranks in 5D, 10D, and 30D benchmark tasks when . We can conclude that best-Cr is the winner for unimodal benchmark task, and mixture is the winner for the multimodal benchmark task. It is the same observations from Wilcoxon signed-rank test of mean values. However, for the population setting as groups, best-jDE and rand-jDE are the winners. It is primarily due to the population diversity increasing the optimization capability of DE algorithm, whose optimization capability is decided by the differential vector production.
In evaluating the critical difference of mean rank scores of each algorithm, we apply an additional Bonferroni-Dunn test on the results of the Friedman tests (Figure 1). In each comparison group, we take the algorithm with the least ranking as a control algorithm. We can make the conclusions that our proposed method has a significant difference with canonical DE (even with jDE in some significant levels). And the mixture method, rand-Cr method, and rand-Cr method are the winner ones for 5D, 10D, and 30D benchmark tasks in setting groups, respectively. Except 30D benchmark functions with setting, there is not any significant difference in setting groups with the significant level of . The exception case indicates that Nash strategy equilibrium implementation is not the best solution concept in our case study, that is, DE algorithm.
5.5. Nash Equilibrium and Mechanism Design of Evolutionary Computation
By considering EC optimization as an algorithmic mechanism design problem, we find a way to describe EC and establish its formal framework. There are several issues that need to be further discussed and investigated.
5.5.1. Domain of Agent Preference
Information is critical element in a game and mechanism design problem. It decides the next response that will be played by agents. Fitness, fitness landscape, and other preset algorithm evaluation metrics can be involved as information in mechanism design of EC algorithm. There are some principles that should be followed when introducing information for each individual. The first is that the information is easy to be obtained with less computational cost, that is, easily retrievable. The second is that the information can support useful element or factor for individual to operate correct action. The third is that the information should be explainable to individual between the action and corresponding information.
For mechanism design in conventional economic field, known as principal and agent theory, it distinguishes different roles, such as principal and agent, in its framework. Agent in game and mechanism design should be rational and self-interested. However, in mechanism design of EC algorithm, it does not need to distinguish these different roles. We consider these individuals as equal participants in the EC, but they should follow principles of rationality and self-interest. It means there must be some reasonable relationship between their action and the information they obtain.
Domain of agent preference is one of the properties of mechanism. In this paper, we try to implement an NE strategy in designed EC algorithm but ignore the domain of agent preference issue that defines choice and payment of an agent. We simplify the agent with an assumption of quasilinear preference (7). We will continue to discuss and investigate other domains of agent preference in EC algorithm design.
In mechanism design of EC algorithm, the information of individual can be the same as expected payoff function. So it is easy to reduce design element when we implement a mechanism design of EC algorithm. We can use the fitness improvement information as the payoff (utility) of each agent in our NE strategy-based DE in this paper. However, payoff concept supports us with more possibility to implement different mechanism design. We will consider this design element in our future work to investigate the algorithmic mechanism design and implementation of EC algorithm.
5.5.2. Solution Concept
The solution concept is important to the mechanism implementation of a game. As we described in Section 3, the objective of the mechanism design of EC algorithm is to design a set of possible strategies and outcome rules for implementing a social choice function in the form of a solution concept. Some solution concepts, such as Nash equilibrium, Bayesian-Nash equilibrium, and dominant equilibrium, can be concrete implementations. Definitely, the stronger the solution concept, the better, because it makes less assumption on an agent. In this study, we take DE as an example to implement an EC algorithm mechanism design by using the weak solution concept (i.e., Nash equilibrium). We will investigate the possibility of other solution concepts’ implementation and the relationship between equilibrium solution implementation and EC optimization performance in the future.
In our mechanism design for an EC algorithm, we found an interesting property in that the NE solution is a dominant solution in established mechanism design. Because the influence of each individual’s improvement is decided by their position and the differential vectors they made in past generation, the fitness improvement information (equal to payoff) of one group is a constant value for all the strategies of the other groups. So the payoff matrix can be expressed as in Table 1.
We prove this conclusion with a case study from the viewpoint of combinatorial theory. Suppose that there is a game on mechanism design of EC algorithm with individuals, and each individual has strategies (). The total number of payoff combinations is given by (14). However, in our designed mechanism design of EC algorithm, the strategy of every individual has only one payoff whatever other individuals play strategies. The total number of payoff combinations of our proposal is given by (15). This means every possible combination of payoff can appear, so within this there must be found the combination with all the elements that is the maximum value, that is, a Nash equilibrium solution, as well as a dominant solution. We have
5.6. Pareto Optimal Solution and Evolutionary Computation Convergence
In game theory, mechanism implements a social choice function, and strategy profile is an equilibrium solution to the game induced by the mechanism. In EC, if we implement a strategy profile following certain equilibrium solutions to achieve Pareto optimal solution, it means the optimization capability of the EC algorithm achieves its maximum state and cannot obtain further better results any more. In this sense, EC convergence is related with this state; that is, the strategy equilibrium solution of EC implements a Pareto optimal solution. We can discuss the EC convergence issue under this assumption.
Social choice function is the Pareto optimal if for every and all types , . This is a formal expression of Pareto solution. The mechanism design of EC implements an equilibrium solution (such as Nash strategy equilibrium in this paper). The desired target of mechanism design for EC is to find the optimal solution of an optimized problem. If the equilibrium solution can guarantee the Pareto optimal of an EC implementation, that is, for every individual , , the convergence state of an EC algorithm can be achieved in this sense. We can establish a formal framework to study on EC convergence based on the equilibrium solution implementation issue. On the other hand, If the equilibrium solution cannot guarantee a Pareto solution, we also need to implement Pareto optimal solution of EC so as to achieve the maximal optimization capability of the algorithm. In this sense, the issues of EC convergence and optimization capability can be transformed as a Pareto solution implementation issue in the algorithmic mechanism design of EC.
6. Conclusion and Future Work
In this paper, we proposed a formal EC framework by considering it as a game or mechanism design problem. The individuals in EC are designed as rational, self-interested agents within this framework and can play using their own search schemes with certain information defined in the game. We designed this framework by defining agent, type, strategy, payoff, and other concepts in EC algorithm. As the first step, we simplified the EC algorithm mechanism design problem to find the Nash strategy equilibrium of fitness improvement as the target of the problem under an assumption of a quasilinear preference of an agent and tried to find the appropriate strategy to achieve this objective. A Nash strategy equilibrium-based DE algorithm considering NE strategy of scale factor , NE strategy of crossover rate , NE strategy of mutation, and a mixture of these was initially implemented, analysed, and discussed. We also came to the interesting conclusion that Nash equilibrium solution is also a dominant solution in our designed algorithm.
The motivation and final objective of this study was to bring the fundamental aspects of game theory and algorithmic mechanism design into the EC field. The design, enhancement, and acceleration of an EC algorithm can therefore be treated as an algorithmic mechanism design problem. This aspect is an original contribution made by this paper. In addition, there are several research subjects that came to light and deserve further investigation. The first is to establish an EC algorithm-related entity mechanism design theory by considering the specific characteristics of evolutionary search, stochastic optimization, and metaheuristics. The second is to design more efficient and effective EC algorithms by introducing a variety of EC implementations. The third is to investigate basic implementation issues with regard to information, agent, type, and strategy in game theory and mechanism design for EC algorithm design. The fourth is to analyse and investigate the relationship between the solution concept implementation and EC convergence and compare the related work with proposals we introduced, such as [26, 27]. These are some of the topics and subjects we will examine in our future research.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.