Complexity

Volume 2017, Article ID 7259032, 14 pages

https://doi.org/10.1155/2017/7259032

## Evolutionary Network Games: Equilibria from Imitation and Best Response Dynamics

^{1}IMT School for Advanced Studies, 55100 Lucca, Italy^{2}Istituto dei Sistemi Complessi (ISC-CNR), 00185 Rome, Italy^{3}Universidad Carlos III, Leganés, 28911 Madrid, Spain

Correspondence should be addressed to Giulio Cimini; ti.accultmi@inimic.oiluig

Received 6 April 2017; Accepted 17 July 2017; Published 24 August 2017

Academic Editor: Tommaso Gili

Copyright © 2017 Giulio Cimini. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

We consider games of strategic substitutes and complements on networks and introduce two evolutionary dynamics in order to refine their multiplicity of equilibria. Within mean field, we find that for the best-shot game, taken as a representative example of strategic substitutes, replicator-like dynamics does not lead to Nash equilibria, whereas it leads to a unique equilibrium for complements, represented by a coordination game. On the other hand, when the dynamics becomes more cognitively demanding, predictions are always Nash equilibria: for the best-shot game we find a reduced set of equilibria with a definite value of the fraction of contributors, whereas, for the coordination game, symmetric equilibria arise only for low or high initial fractions of cooperators. We further extend our study by considering complex topologies through heterogeneous mean field and show that the nature of the selected equilibria does not change for the best-shot game. However, for coordination games, we reveal an important difference: on infinitely large scale-free networks, cooperative equilibria arise for any value of the incentive to cooperate. Our analytical results are confirmed by numerical simulations and open the question of whether there can be dynamics that consistently leads to stringent equilibria refinements for both classes of games.

#### 1. Introduction

Strategic interactions among individuals located on a network, be it geographical, social, or of any other nature, are becoming increasingly relevant in many economic contexts. Decisions made by our neighbors on the network influence ours and are in turn influenced by their other neighbors to whom we may or may not be connected. Such a framework makes finding the best strategy a very complex problem, almost always plagued by a very large multiplicity of equilibria. Researchers are devoting much effort to this problem, and an increasing body of knowledge is being consolidated [1–3]. In this work we consider games of strategic substitutes and strategic complements on networks, as discussed in [4]. In this paper, Galeotti et al. obtained an important reduction in the number of game equilibria by going from a complete information setting to an incomplete one. They introduced incomplete information by assuming that each player is only aware of the number of neighbors he/she has, but not of their identity nor of the number of neighbors they have in turn. We here aim at providing an alternative equilibrium refinement by looking at network games from an evolutionary viewpoint. In particular, we look for the set of equilibria which can be accessed according to two different dynamics for players’ strategies and discuss the implications of such reduction. Furthermore, we go beyond the state-of-the-art mean field approach and consider the role of complex topologies with a heterogeneous mean field technique.

Our work belongs to the literature on strategic interactions in networks and its applications to economics [5–13]. In particular, one of the games we study is a discrete version of a public goods game proposed by Bramoullé and Kranton [14], who opened the way to the problem of equilibrium selection in this kind of games under complete information. Bramoullé further considered this problem [15] for the case of anticoordination games on networks, showing that network effects are much stronger than for coordination games. As already stated, our paper originates from Galeotti et al. [4], for they considered one-shot games with strategic complements and substitutes and model equilibria resulting from incomplete information. Our approach is instead based on evolutionary selection of equilibria—pertaining to the large body of work emanating from the Nash programme [16–19]—and is thus complementary to theirs. In particular we focus on the analysis of two evolutionary dynamics (see Roca et al. [20] for a review of the literature) in two representative games and on how this dynamics leads to a refinement of the Nash equilibria or to other final states. The dynamics we consider are Proportional Imitation [21, 22], which does not lead in general to Nash equilibria, and best response [23, 24], which instead allows for convergence to Nash equilibria—an issue about which there are a number of interesting results in the case of a well-mixed population [25–27]. As we are working on a network setup, our specific perspective is close to that of Boncinelli and Pin [28]. They elaborate on the literature on stochastic stability [19, 29] (see [24, 30] for an early example of related dynamics on lattices) as a device that selects the equilibria that are more likely to be observed in the long run, in the presence of small errors occurring with a vanishing probability. They work from the observation [31] that different equilibria can be selected depending on assumptions on the relative likelihood of different types of errors. Thus, Boncinelli and Pin work with a best response dynamics and by means of a Markov Chain analysis find, counterintuitively, that when contributors are the most perturbed players, the selected equilibrium is the one with the highest contribution. The techniques we use here are based on differential equations and have a more dynamical character, and we do not incorporate the possibility of having special distributions of errors—although we do consider random mistakes. Particularly relevant to our work is the paper by López-Pintado [32] (see [33] for an extension to the case of directed networks) where a mean field dynamical approach involving a random subsample of players is proposed. Within this framework, the network is dynamic, as if at each period the network were generated randomly. Then a unique globally stable state of the dynamics is found, although the identities of free riders might change from one period to another. The difference with our work is that we do not deal with a time-dependent subsample of the population, but we use a global mean field approach (possibly depending on the connectivity of individuals) to describe the behavior of a static network.

In the remainder of this introduction we present the games we study and the dynamics we apply for equilibrium refinement in detail, discuss the implications of such a framework on the informational settings we are considering, and summarize our main contributions.

##### 1.1. Framework

###### 1.1.1. Games

We consider a finite set of agents of cardinality , linked together in a fixed, undirected, exogenous network. The links between agents reflect social interactions, and connected agents are said to be “neighbors.” The network is defined through a symmetric matrix with null diagonal, where means that agents and are neighbors, while means that they are not. We indicate with the set of ’s neighbors; that is, , where the number of such neighbors is the* degree* of the node.

Each player can take one of two actions , with denoting ’s action. Hence, only pure strategies are considered. In our context (particularly for the case of substitutes), action 1 may be interpreted as* cooperating* and action 0 as not doing so—or* defecting*. Thus, the two actions are labeled in the rest of the paper as and , respectively. There is a cost , where , for choosing action , while action bears no cost.

In what follows we concentrate on two games, the best-shot game and a coordination game, as representative instances of strategic substitutes and strategic complements, respectively. We choose specific examples for the sake of being able to study analytically their dynamics. To define the payoffs we introduce the following notation: is the aggregate action in and .

*(a) Strategic Substitutes: Best-Shot Game*. This game was first considered by Bramoullé and Kranton [14] as a model of the local provision of a public good. As stated above, we consider the discrete version, where there are only two actions available, as in [4, 28]. The corresponding payoff function takes the formwhere is the Heaviside step function if and otherwise.

*(b) Strategic Complements: Coordination Game*. For our second example, we follow Galeotti et al. [4] and consider again a discrete version of the game, but now let the payoffs of any particular agent be given byAssuming that , we are faced with a coordination game where, as discussed in [4], depending on the underlying network and the information conditions, there can generally be multiple equilibria.

###### 1.1.2. Dynamics

Within the two games we have presented above, we now consider evolutionary dynamics for players’ strategies. Starting at with a certain fraction of players randomly chosen to undertake action , at each round of the game players collect their payoff according to their neighbors’ actions and the kind of game under consideration. Subsequently, a fraction of players update their strategy. We consider two different mechanisms for strategy updating.

*(a) Proportional Imitation (PI) [21, 22]*. It represents a rule of imitative nature in which player may copy the strategy of a selected counterpart , which is chosen randomly among the neighbors of . The probability that copies ’s strategy depends on the difference between the payoffs they obtained in the previous round of the game:where is a normalization constant that ensures and allows for mistakes (i.e., copying an action that yielded less payoff in the previous round). Note that because of the imitation mechanism of PI, the configurations and are absorbing states: the system cannot escape from them and not even mistakes can reintroduce strategies, as they always involve imitation. On the other hand, it can be shown that PI is equivalent to the well-known replicator dynamics in the limit of an infinitely large, well-mixed population (equivalently, on a complete graph) [34, 35]. As was first put by Schlag [22], the assumption that agents play a random-matching game in a large population and learn the actual payoff of another randomly chosen agent, along with a rule of action that increases their expected payoff, leads to a probability of switching to the other agent’s strategy that is proportional to the difference in payoffs. The corresponding aggregate dynamics is like the replicator dynamics. See also [36] for another interpretation of these dynamics in terms of learning.

*(b) Best Response (BR)*. This rule was introduced in [23, 24] and has been widely used in the economics literature. BR describes players that are rational and choose their strategy (myopically) in order to maximize their payoff, assuming that their neighbors will again do what they did in the last round. This means that each player , given the past actions of their partners , computes the payoffs that he/she would obtain by choosing action 1 (cooperating) or 0 (defecting) at time , respectively, and . Then actions are updated as follows:and if . Here again represents the probability of making a mistake, with indicating fully rational players.

The reason to study these two dynamics is because they may lead to different results as they represent very different evolutions of the players’ strategies. In this respect, it is important to mention that, in the case , Nash equilibria are stable by definition under BR dynamics and, vice versa, any stationary state found by BR is necessarily a Nash equilibrium. On the contrary, with PI this is not always true: even in the absence of mistakes, players can change action by copying better-performing neighbors, also if such change leads to a decreasing of their payoffs in the next round. Another difference between the two dynamics is the amount of cognitive capability they assume for the players: whereas PI refers to agents with very limited rationality, which imitate a randomly chosen neighbor on the only condition that he/she does better, BR requires agents with a much more developed analytic ability.

###### 1.1.3. Analytical and Informational Settings

We study how the system evolves by either of these two dynamics, starting from an initial random distribution of strategies. In particular, we are interested in the global fraction of cooperators and its possible stationary value . We carry out our calculations in the framework of a homogeneous mean field (MF) approximation, which is most appropriate to study networks with homogeneous degree distribution like Erdös-Rényi random graphs [37]. The basic assumption underlying this approach is that every player interacts with an “average player” that represents the actions of his/her neighbors. More formally, the MF approximation consists in assuming that when a player interacts with a neighbor of theirs, the action of such a neighbor is with probability (and otherwise), independently of the particular pair of players considered [38]. Loosely speaking, this amounts to having a very incomplete information setting, in which all players know only how many other players they will engage with, and is reminiscent of that used by Galeotti et al. [4] for their refinement of equilibria. However, the analogy is not perfect and therefore, for the sake of accuracy, we do not dwell any further on the matter. In any case, MF represents our setup for most of the paper.

As an extension of the results obtained in the above context, we also study the case of highly heterogeneous networks, that is, networks with broad degree distribution , such as scale-free ones [39]. In these cases in fact there are a number of players with many neighbors (“hubs”) and many players with only a few neighbors, and this heterogeneity may give rise to very different behaviors as compared to Erdös-Rényi systems. Analytically, this can be done by means of the heterogeneous mean field technique (HMF) [40] which generalizes, for the case of networks with arbitrary degree distribution, the equations describing the dynamical process by considering degree-block variables grouping nodes within the same degree. More formally, now when a player interacts with a neighbor of theirs, the action of such a neighbor is with probability (and otherwise) if is the neighbor’s degree ( is the density of cooperators within players of degree ). By resorting to this second perspective we are able to gain insights on the effects of heterogeneity on the evolutionary dynamics of our games.

##### 1.2. Our Contribution

Within this framework, our main contribution can be summarized as follows. In our basic setup of homogeneous networks (described by the mean field approximation): for the best-shot game, PI leads to a stationary state in which all players play , that is, to full defection, which is however non-Nash as any player surrounded by defectors would obtain higher payoff by choosing cooperation (at odds with the standard version of the public goods game). This is the result also in the presence of mistakes, unless the probability of errors becomes large, in which case the stationary state is the opposite, , that is, full cooperation, also non-Nash. Hence, PI does not lead to any refinement of the Nash equilibrium structure. On the contrary, BR leads to Nash equilibria characterized by a finite fraction of cooperators , whereas, in the case when players are affected by errors, this fraction coincides with the probability of making an error as the mean degree of the network goes to infinity. The picture is different for the coordination game. In this case, PI does lead to Nash equilibria, selecting the coordination in 0 below a threshold value of and the opposite state otherwise. This threshold is found to depend on the initial fraction of players choosing . Mistakes lead to the appearance of a new possibility, an intermediate value of the fraction of players choosing 1, and as before the initial value of this fraction governs which equilibrium is selected. BR gives similar results, albeit for the fact that a finite fraction of 1 actions can also be found even without mistakes, and with mistakes the equilibria are not full 0 or 1 but there is always a fraction of mistaken players. Finally, changing the analytical setting by proceeding to the heterogeneous mean field approach does not lead to any dramatic change in the structure of the equilibria for the best-shot game. Interestingly, things change significantly for coordination games—when played on infinitely large scale-free networks. In this case, which is the one where the heterogeneous mean field should make a difference, equilibria with nonvanishing cooperation are obtained for any value of the incentive to cooperate (represented by the parameter ).

The paper is organized in seven sections including this introduction. Section 2 presents our analysis and results for the best-shot game. Section 3 deals with the coordination game. In both cases, the analytical framework is that of the mean field technique. After an overall analysis of global welfare performed in Section 4, Section 5 presents the extensions of the results for both games within the heterogeneous mean field approach, including some background on the formalism itself. Finally, Section 6 contains an assessment of the validity of all these analytical findings in light of the results of recent numerical simulations of the system described above, and Section 7 concludes the paper summarizing our most important findings concerning the refinement of equilibria in network games and pointing to relevant open questions.

#### 2. Best-Shot Game

##### 2.1. Proportional Imitation

We begin by considering the case of strategic substitutes when imitation of a neighbor is only possible if he/she has obtained better payoff than the focal player; that is, in (3). In that case, the main result is the following.

Proposition 1. *Within the mean field formalism, under PI dynamics, when the final state for the population is the absorbing state with a density of cooperators (full defection) except if the initial state is full cooperation.*

*Proof. *Working in a mean field context means that individuals are well-mixed, that is, every player interacts with average players. In this case the differential equation for the density of cooperators isThe first term is the probability of picking a defector with a neighboring cooperator, times the probability of imitation . The second term is the probability of picking a cooperator with a neighboring defector, times the probability of imitation . In the best-shot game a defector cannot copy a neighboring cooperator (who has lower payoff by construction), whereas, a cooperator eventually copies one of his/her neighboring defectors (who has higher payoff). Hence and is equal to the payoff difference . Since the normalization constant for strategic substitutes, (5) becomesThe solution, for any initial condition , ishence for : the only stationary state is full defection unless .

*Remark 2. *As discussed above, PI does not necessarily lead to Nash equilibria as asymptotic, stationary states. This is clear in this case. For any the population ends up in full defection, even if every individual player would be better off by switching to cooperation. This phenomenon is likely to arise from the substitutes or anticoordination character of the game: in a context in which it is best to do the opposite of the other players, imitation does not seem the best way for players to decide on their actions.

Proposition 3. *Within the mean field formalism, under PI dynamics, when the final state for the population is the absorbing state (full defection) when , when , and when . When the initial state is or , it remains unchanged.*

*Proof. *Equation (5) is still valid, with unchanged, whereas, . By introducing the effective cost we can rewrite (7) asHence for only for () and instead for () then , and for () then for (cooperation is favored now).

*Remark 4. *As before, PI does not drive the population to a Nash equilibrium, independently of the probability of making a mistake. However, mistakes do introduce a bias towards cooperation and thus a new scenario: when their probability exceeds the cost of cooperating, the whole population ends up cooperating.

##### 2.2. Best Response

We now turn to the case of the best response dynamics, which (at least for ) is guaranteed to drive the system towards Nash equilibria. In this scenario, we have not been able to find a rigorous proof of our main result, but we can make some approximations in the equation that support it. As we will see, our main conclusion is that, within the mean field formalism under BR dynamics, when the final state for the population is a mixed state , , for any initial condition.

Indeed, for BR dynamics without mistakes, the homogeneous mean field equation for iswhere the first term is the probability of picking a cooperator who would do better by defecting, and the second term is the probability of picking a defector who would do better by cooperating. This far, no approximation has been made; however, these two probabilities cannot be exactly computed and we need to estimate them.

To evaluate the two probabilities, we can recall that always, whereas when none of the neighbors cooperates and otherwise. Therefore, for an average player of degree we have that . Consistently with the mean field framework we are working on, as a rough approximation, we can assume that every player has degree (the average degree of the network), so that . Thus, we haveTo go beyond this simple estimation, we can work out a better approximation by integrating over the probability distribution of players’ degrees . For Erdös-Rényi random graphs, in the limit of large populations (), it is . This leads to and, subsequently,

*Remark 5. *The precise asymptotic value for the density of cooperators, , depends on the approximation considered above. However, at least for networks that are not too inhomogeneous, the approximations turn out to be very good, and therefore the corresponding picture for the evolution of the population is quite accurate. It is interesting to note that, whatever its exact value, in both cases is such that the right-hand sides of (10) and (11) vanish and, furthermore, is an attractor of the dynamics, because .

How is the above result modified by mistakes? When , (9) becomeswhere the first term accounts for cooperators rightfully switching to defection and defectors wrongly selecting cooperation, while the second term accounts for defectors correctly choosing cooperation and cooperators mistaken to defection. Proceeding as before, and again in the limit , we approximate , thus arriving atfrom which it is possible to find the attractor of the dynamics . Such attractor in turn exists if a threshold that is bounded below by 1/2, which would be tantamount to players choosing their action at random. Therefore, all reasonable values for the probability of errors allow for equilibria.

*Remark 6. *To gain some insight on the cooperation levels arising from BR dynamics in the Nash equilibria, we have numerically solved (13). The values for are plotted in Figure 1 for different values of , as a function of . We observe that the larger the , the lower the cooperation level. The intuition behind such result is that the more the connections that every player has, the lower the need to play 1 to ensure obtaining a positive payoff. It could then be thought that this conclusion is reminiscent of the equilibria found for best-shot games in [4], which are nonincreasing in the degree. However, this is not the case, as in our work we are considering an iterated game that can perfectly lead to high degree nodes having to cooperate. Note also that this approach leads to a definite value for the density of cooperators in the Nash equilibrium, but there can be many action profiles for the player compatible with that value, so multiplicity of equilibria is reduced but not suppressed.