Abstract
The cooperative game of global temperature lacks automaticity and emotional jamming. To solve this issue, an agent-based modelling method is developed based on Milinski’s noncooperative game experiments. In addition, genetic algorithm is used to improve the investment strategy of each agent. Simulations are carried out by designing different coding schemes, mutation schemes, and fitness functions. It is demonstrated that the method can achieve maximum benefits under the premise of the agent non-cooperative game through encouraging optimal individuals. The results provide a sound basis for developing tools and methods to support the simulation of climate game strategy that involves multiple stakeholders.
1. Introduction
Climate change is a global issue that is addressed by taking into account various factors in society. Aiming at a complex social game issue, a large number of participants should make effort to prevent global climate variation. Global climate cooperation is proved to be very necessary in the climate game [1]. A comprehensive research of climate game is a challenging topic.
Global greenhouse gas (GHG) emissions have been growing greatly [2, 3]. Humankinds are facing a dramatic change of living conditions on the earth when the already-rising global temperature passes a certain threshold [4–8]. To reduce the risk of dangerous climate change, it needs to take the main GHG emissions countries into a “climate coalition,” which provides climate ambitious emission reduction at the least cost [9]. At the same time, the broad alliance may reflect strongly to realize the incentive to “free ride” [10]. Therefore, GHG emission is not a problem for a single country. In other words, no country can solve the global climate change problem by acting alone. States have to cooperate in order to address the threat of climate change.
Although there are a few game-theoretical works on climate change, in general the social dilemma situations and the public goods game are perfectly fine models for the problem too. And these games have a very rich history as agent-based simulations. The most famous mathematical metaphor for a social dilemma denotes the prisoner’s dilemma [11, 12]. Other well-studied models include public goods games [13, 14], which essentially represent a generalization of the pairwise prisoner’s dilemma to interactions in groups of arbitrary size [15, 16]. Foremost, there is the issue of reward and punishment, which has recently been studied a lot in order to understand how these two basic social forces may avert the dilemma [17–20]. Then there are also works concerning the critical mass, conditional strategies, population density, heterogeneity, and interdependent networks in social dilemmas. These subjects have been studied extensively in the very recent past, and their implications for the resolution of social dilemmas such as the climate change dilemma as agent-based models are very significant [21–25]. Most recently, the shift to agent-based modeling has also been highlighted for human bargaining [26], which is obviously of relevance for the climate change dilemma as their players (countries, nations) have to agree on a certain policy that they will then carry out.
In essence, global climate game is the process of competition between the participants with different game strategies [27–31]. They must balance the relationship between economic development and environmental protection by finding the Nash equilibrium between these two interests. In other words, only in content with optimality condition and Nash equilibrium of the dual requirement, the climate cooperation will be the most stable and efficiency international cooperation.
Climate protection programs that appeal to a human sense of fairness, that is, each player contributes a “fair share” to the collective goal, are more likely to avoid irrational self-detrimental behavior [32]. Milinski et al. [33] proposed the collective-risk social dilemma as a framework for investigating the inherent problems of avoiding dangerous climate change, and performed simulation to study the game. According to Milinski’s experiment, students invest anonymously, with each student being informed of the cumulative investment sum after each round. Under such circumstance, the trade-off between personal benefits and group interests provides a basis for each student to make its investment scheme. Moreover, students can learn from a successful scheme to make better decisions in the next round. However, this strategy, in which computer runs the dice program, is a kind of the random investment strategy. A stochastic process is one whose behavior is nondeterministic, and subsequent of investment is determined by a random element. Therefore, it is more difficult to obtain the optimal investment strategy.
All those works can be linked to the voluntary contribution games with punishment possibility. The final step of the complex game process should be built by several models, algorithms, and different experiments application in order to make it clear and systematic. This is also the next stage of this work.
The aim of this paper focuses on developing evolutionary model and simulation strategy to improve Milinski’s investment strategy. The investment strategy using genetic algorithm (GA) of agent-based modeling is established based on the noncooperative game experiments in [33]. Firstly, used agents represent climate game players to develop the game modeling. Then, we use GA for investment strategy simulation. The GA investment strategy is specifically designed to support the study of multiparticipant climate change game using computational modeling, simulation, programming, and running. GA is a population-based biomimetic evolutionary method to solve various complex decision-making problems [34–36]. This heuristic is routinely used to generate useful solutions for optimization and search problems. In this work, the climate game problem is regarded as an optimization problem, and thus, optimal investment scheme for each agent will be obtained through GA.
2. Agent-Based Modeling of the Climate Game Problem
Climate change is a classic instance of complex, bottom-up, and multiagent human behavior. Using game theory to research on global climate change is an effective way for climate cooperation. Each agent represents a student or a commonwealth of them. The essence of climate game is the process of competition between the agents’ strategies. Each agent must balance between economic development and environmental protection, trying to find the Nash equilibrium between these two interests.
In [33], thirty groups of six agents took part in a climate game where each agent needs to build a data account to store its amount of investment. At the beginning, the initial balance of each agent’s account is €40. Subsequently, each agent must invest €2 in each round of game, and all six agents must invest an average of €120 in 10 rounds. Only in this way, it can be possible to ensure that the finally total investment is more than or equal to €120. At the same time, each agent in each round is required to invest only €0, €2, or €4. Therefore, this section will discuss the strategy of the random selection of €0, €2, or €4 in each investment, that is, it chooses a value with a certain probability.
According to Milinski’s experiment, half of the groups succeed in reaching the target sum, whereas the others only marginally fail. When the risk of loss was only as high as the necessary average investment or even lower, the groups generally failed to reach the target sum. It was shown that the investment sum of about half of the agents was not less than €120, and in the most reasonable way, each agent invested €2 at each round in average. Therefore, the possibility of investing €2 was twice that of investing other numbers. It can be seen on this basis that the probability of providing €2 is 0.5, $0 with the probability 0.25, and $4 with the probability 0.25.
3. Genetic Algorithm for the Climate Game Problem
To improve investment strategy, an evolutionary strategy based on GA is proposed in this paper. Specifically, the focus is on agent’s decision-making process through simulation experiment. A GA-based model for the climate game problem is developed, and its details are described as follows.(i)Each player is viewed as an autonomous agent.(ii)Each agent establishes and maintains a database.(iii)Each agent’s investment sum in all 10 rounds will be stored in the database.(iv)The contribution of each agent is determined by the GA-based strategy.(v)Individual code: each agent’s investment in 10 rounds and each agent’s investment sequence as an individual of GA.(vi)Integer code: 0, 2, 4. For example, the first agent in 10 rounds can be encoded as 2222222222.
The main program based on GA is illustrated in Figure 1. In the figure, means that an agent has a probability of 90% to lose all savings if the target sum €120 is not reached. The meaning of and is that the probabilities of losing all their money are 50% and 10%, respectively, if €120 is not reached.
The meanings of other marks are explained as follows:(i) represents the investment amount of the th agent;(ii) represents the total remaining saving in the account of the th agent;(iii) represents the total remaining saving in the account of the th agent;(iv) are the random variables;(v) represents the punishment probability;(vi), and , are the loop variables.
In initialization, we set the population size as 20 and randomly generate 20 individuals as the initial population to represent 20 investment schemes. In order to contrast, we save the generated 20 investment schemes into the corresponding records in the database established.
3.1. Random Investment
The random investment subroutine is shown in Figure 2 where each agent is provided €40 at the beginning. After that, the random subroutine will run to simulate the investment activities of the six agents for 10 rounds, then transfer the generated data to the punishment subprogram. The investment results and the incomes of each agent will be stored in the database at last.
3.2. GA Investment
By running the GA investment subroutine for each agent, a new investment scheme can be generated, from which the total amount of money can be obtained. Then, the personal benefit for each agent after experiencing the risk of losing all the remaining money can be calculated. The investment scheme for each agent, the total amount of money that each agent invested, and the cumulative investment sum among all these agents will be saved into the database. As such, there are 21 records of investment in the database.
Figure 3 is the process of the simple GA investment subroutine. is a randomly generated real number, and the condition implies that the mutation probability of variation is 0.1. The process is explained as follows.
Step 1. Rank the fitness values of the investment schemes (which are decoded as individuals in GA) in descending order.
Step 2. If the random number , the best individual would mutate using single-point mutation; otherwise, go to Step 3.
Step 3. If the random number , there is no mutation in the game. The best individual is selected as the new one directly.
Step 4. When iteration >100, end the execution; otherwise, return.
3.2.1. Individual Representation
As discussed earlier in this paper, a database is established for each agent to save its investment record (which includes its investment quota and its remaining money in each round). In this case, the cumulative investment sums for agents can be obtained. For instance, if the investment record of the th agent during 10 rounds is 2220044220, which means that the agent invests €2 in the first three rounds, €0 in the fourth and fifth rounds, and so on. We can get that the total amount of money invested by the th agent throughout 10 rounds reaches €18. If the cumulative investment sum among agents during 10 rounds is €110, and the th agent is chosen to be punished since the target sum (€120) has not been achieved, the th agent will lose its remaining €22. In this way, we put data 2220044220, 22, and 110 in the database for the th agent.
The integer coding method is used in GA, with each individual representing the investment quotas in 10 rounds for each agent. An individual can be 2220044220 for the th agent in the example mentioned in the last paragraph.
3.2.2. Fitness Function
Individuals in GA are evaluated via the fitness function. Since the goal is to achieve the maximization of personal benefits and cumulating the investment sum, the fitness function is designed as follows:
In the equation, refers to the agent index, and refers to the investment scheme index. denotes the fitness value of the th scheme for the th agent, refers to the remaining saving of the th agent via its th scheme, that is, profits after the punishment which is 90% probability to punish if the target sum has not been reached, and means the cumulative investment sum from all agents involved. The weighting coefficient reflects the balance between individual benefits and cumulative investment sum.
All the investment schemes are ranked in descending order in terms of fitness value. Consequently, the best investment scheme with the largest fitness value can be obtained, and it can be directly established as the designated scheme for the next game or established after an appropriate adjustment.
3.2.3. Mutation
In the GA investment subroutine, mutation is performed for the adjustment of the best investment scheme with a probability of 0.9. Specifically, a real number (within ) is randomly generated. If , the best investment scheme is directly established as the designated scheme for the next game; otherwise, perform the mutation.
Site-based mutation method is adopted, which can be described as follows: randomly select a gene from the individual and replace it with randomly generated numbers 0, 2, or 4, and a new scheme is obtained. The mutation probability can be 0.1 or other values between (0,1). Through the GA investment subroutine, an agent can get a recommended option as a reference scheme.
3.3. Punishment
According to [28], the subroutine of random investment with punishment is shown in Figure 4. After 10 rounds, if an agent’s investment sum achieves or exceeds €120, it will obtain the surplus in its account. Otherwise, it will enter the punishment subroutine. The routine will punish all agents through throwing dice, leading to that all agents have 90%, 50%, or 10% probabilities to lose their surpluses. This case will be discussed as follows.
In the punishment subroutine, for every agent, the computer produces a random number , and if is greater than (, 0.5, or 0.1), the step will be skipped, and the surplus money in the account will be saved. Otherwise, all surplus money in the account will be confiscated.
3.4. Fitness Calculation and Sorting
This stage involves calculating the fitness value of 21 individuals in the database, ranking these individuals in terms of fitness value and abandoning the one with the minimum fitness value. So there are still 20 records of investment in the database. Then it moves to the next round, until the given number of games (i.e., 100) is completed.
4. Simulations and Results
The results of the simulation include three parts. First, we obtained variation curves of the total remaining savings in all six agents’ accounts under different losing probability and weighting coefficient . Then we also obtained variation curves of cumulative investment sum under varying and , and finally, we identified the relationship between the variables and the parameters.
4.1. Parameter Set 1
We first carry out the experiments to study the total remaining savings and the cumulative investment sum with and varying . By implementing the experiment with different values of the parameters, variation curves of total remaining savings in all six agents’ accounts are shown in Figures 5(a)–5(d), where denotes the punishment probability if the target sum is not reached, and refers to the weighting coefficient in the equation.
(a)
(b)
(c)
(d)
There are two observations from the above figures under the 90% treatment: (1) the total remaining savings for all the six agents in a group decrease in response to the growth of the weighting coefficient ; (2) the total remaining savings remain as €120 when . Consequently, the total remaining savings increase when coefficient declines, showing a bias towards the group interests. On the other hand, personal benefits are more inclined to be accomplished, and the total remaining savings decrease when a higher value of is used. The group interests and personal benefits are well balanced when , where the total remaining saving remains as €120 and the Nash equilibrium is achieved. Variation curves of the cumulative investment sum among all the six agents in a group under different values of in the 90% treatment are illustrated in Figure 6.
Two observations can be obtained from Figure 6: (1) the cumulative investment sum among all the six agents in a group decreases in response to the growth of the weighting coefficient ; (2) the cumulative investment sum remains as €120 when . The results indicate that the fitness function designed in (3.1) inclines toward the group interests, and that the cumulative investment sum increases as decreases. The results also show a bias towards personal benefits, and the cumulative investment sum decreases when a higher value of is used. When , the group interests and personal benefits are both well taken care of, and the cumulative investment sum is about €120.
Although the experiment is in the 90% treatment (i.e., ), the above conclusions are also applicable for cases in the 50% treatment (i.e., ) and 10% treatment (i.e., ).
4.2. Parameter Set 2
We then carry out the experiments to study the total remaining savings and the cumulative investment sum with and varying . If the target sum €120 is not reached, an agent will risk losing all their remaining money with a probability of either 0.9, 0.5, or 0.1. Results on the total remaining saving among all the 6 agents in a group under different values of in the 50% treatment (i.e., ) are drawn through our experiment, as shown in Figures 7(a)–7(c). Those on the cumulative investment sum are illustrated in Figure 8.
(a)
(b)
(c)
4.3. Parameter Set 3
We further carry out the experiments to study the total remaining savings and the cumulative investment sum with and varying . Results on the total remaining savings and the cumulative investment sum among all the six agents in a group under different values of in the 10% treatment (i.e., ) are shown in Figures 9(a)–9(c) and Figure 10, respectively.
(a)
(b)
(c)
It can be concluded that the cumulative investment sum goes down, and that the total remaining saving increases when loss probability increases. This indicates that a country needs to invest more for recovery if dangerous climate change occurs and causes great damage.
5. Conclusion
An agent-based evolutionary model and a GA-based solution strategy are proposed in this paper. Based on the principle of maximizing individual and collective interests, linear weighting is used for the GA fitness function, and a coding and mutation operator is designed for GA evolutionary optimization strategies. The simulation experiments with groups of six agents show that it can achieve maximum benefits under the premise of the agent noncooperative game through encouraging optimal individuals. The results provide a solid basis for studying climate game strategy using multiagent modelling and simulation. This approach also has the potential to simulate the experiment that contains a large amount of data.
Acknowledgments
This work is supported in part by the National Natural Science Foundation of China (no. 60970021) and the Department of Education Foundation of Zhejiang Province (no. Y201225032). The authors are grateful to Dr. Hywel Williams who gave much advice to them concerning this paper at the University of East Anglia.