Abstract
When confronted with multiple Nash equilibria, decision makers have to refine their choices. Among all known Nash equilibrium refinements, the perfectness concept is probably the most famous one. It is known that weakly dominated strategies of two-player games cannot be part of a perfect equilibrium. In general, this undominance property however does not extend to -player games (E. E. C. van Damme, 1983). In this paper we show that polymatrix games, which form a particular class of -player games, verify the undominance property. Consequently, we prove that every perfect equilibrium of a polymatrix game is undominated and that every undominated equilibrium of a polymatrix game is perfect. This result is used to set a new characterization of perfect Nash equilibria for polymatrix games. We also prove that the set of perfect Nash equilibria of a polymatrix game is a finite union of convex polytopes. In addition, we introduce a linear programming formulation to identify perfect equilibria for polymatrix games. These results are illustrated on two small game applications. Computational experiments on randomly generated polymatrix games with different size and density are provided.
1. Introduction
Interest for game theoretic applications has been growing in engineering, management and political sciences. A polymatrix game is a confrontation of players in a normal and noncooperative context. Polymatrix games form a particular class of -player games. A polymatrix game with players is such that player ’s payoff relative to player ’s decisions is independent from the remaining players’ choices. Considering as the set of all players, each player controls a finite set of pure strategies with . We define .
1.1. Literature Review
The Nash equilibrium concept [1] has often been presented as the most desirable solution for games. Authors like Avis and Fukuda [2] and Audet et al. [3, 4] presented computational methods to enumerate all Nash extreme points for two-player games. Some other authors like Daskalakis et al. [5] and Hazan and Krauthgamer [6] have recently studied the Nash equilibrium computation complexity problem, also for two-player games. Etessami and Yannakakis [7] studied the complexity of computing approximated Nash equilibria for three or more players finite games. Some pioneering results on polymatrix games are to be mentioned. The complementary pivoting method was used by Yanovskaya [8] to compute polymatrix game equilibria. Howson [9], Eaves [10], and Howson and Rosenthal [11] also adopted the same approach. Quintas [12] showed that the set of Nash equilibrium points in a polymatrix game is a finite union of convex polytopes. Miller and Zucker [13] showed how to reduce the polymatrix game equilibria problem to a copositive-plus linear complementarity problem (LCP) solvable with a single application of Lemke’s algorithm [14]. Wilson [15] extended the Lemke and Howson algorithm [16] for finding a Nash equilibrium of a two-player game to -player games. Govindan and Wilson [17] used sequences of polymatrix games to approximate and compute Nash equilibrium for -player games. We addressed the problem of enumeration of all polymatrix game Nash extreme equilibria in Audet et al. [4]. Papadimitriou and Roughgarden [18] showed that computing a correlated equilibrium of a polymatrix game can be done in polynomial time.
1.2. Motivation
Decision makers, confronted to multiple Nash equilibria, have to refine their choices using other rational concepts in addition to the concept of Nash equilibrium. Game theorists introduced many refinement concepts. Among all known Nash equilibrium refinements, the perfectness concept is probably the most studied one. This concept was first introduced by Selten [19] for finite strategic form games. It is based on the idea that a reasonable equilibrium should be stable against slight perturbations in the equilibrium strategies. Hence, the perfect refinement defines stability conditions with respect to slight imperfections of rationality sometimes called “trembling-hand perfection.” Selten [19] and Myerson [20] showed that there is at least one perfect equilibrium for any strategic form game. Selten’s proof of perfect equilibrium existence is indirect and relies on the existence of Nash equilibrium in every perturbed game. Topolyan [21] used a generalization of Kakutani’s fixed point existence theorem to prove the existence of perfect equilibria in finite normal form games and extensive games with perfect recall. Her constructive proof generates a correspondence whose fixed points are precisely the perfect equilibria of a given finite game. For bimatrix games, Borm et al. [22] described a maximal Selten subset as a set of interchangeable perfect equilibria. Each maximal Selten subset is a subset of a maximal Nash subset and each extreme point of a maximal Selten subset corresponds to an extreme perfect equilibrium. Laslier and van der Straeten [23] used the concept of “trembling-hand perfection” to analyze an electoral competition problem under imperfect information. Watanabe and Yamato [24] used the same concept to study a choice of auction in seller cheating. Miltersen and Sørensen [25] proposed a computational method to find quasiperfect Nash equilibria for two-player games. While the perfectness verification problem is known to be easy with two players [26], to our knowledge, no results are reported on the perfect refinement of Nash equilibria for polymatrix games.
In this paper, we intend to set an automatic procedure to verify the perfectness of polymatrix games Nash equilibria. Section 2 recalls the definition of a polymatrix game Nash Equilibrium. Section 3 sets a new characterization for polymatrix games perfect equilibria and proposes a linear programming approach to conclude the perfectness of a Nash equilibrium point. Section 4 states a geometric property on the set of perfect equilibria. Section 5 presents computational results obtained over sets of randomly generated polymatrix games with different size and density.
2. Polymatrix Games Nash Equilibria
Let us define as the payoff matrix of player against all other players. A partial payoff is assigned to player , if player plays his strategy and player plays his strategy . Player ’s partial payoff matrix relative to player ’s strategic decisions is a matrix . The total payoff for player corresponding to any pure strategic choice of the players is
Each player selects a probability vector over his set of pure strategies and tries to maximize his own total payoff. The mixed strategy vector is such that , where for all , is the relative probability with which player plays his strategy . Player ’s mixed strategies belong to the set: where is a row vector with all entries equal to . At the end of the game, the total payoff of player can be expressed as follows:
Like any -player strategic form game, a polymatrix game has at least one Nash equilibrium [1]. We can define a Nash equilibrium to be a -tuple of mixed strategies such that for any other -tuple the following inequality is satisfied: that is, player ’s payoff relative to all other players is simultaneously maximized.
We denote by NE the set of Nash equilibria. This set is the union of a finite number of polytopes called maximal Nash subsets [12]. We define an extreme equilibrium to be any vertex of the maximal Nash subsets. Hence, the set of extreme equilibria is the set of vertices of the maximal Nash subsets. A subset is called a Nash subset if and only if every pair of elements in is interchangeable; that is,
A Nash subset is called maximal if it is not properly contained in another Nash subset [22]. Enumeration of all maximal Nash subsets can be achieved using an algorithm for the enumeration of all maximal cliques of a graph [26].
3. Polymatrix Games Nash Perfect Equilibria
Let be a Nash equilibrium of a polymatrix game with players, and let be the set of pure best replies of player against : where is a column vector with all entries equal to zero, except the entry which equals one. Let us also define , where is the convex envelope of .
3.1. Polymatrix Game Perfect Equilibrium Definition
Using Selten’s definition of perfect equilibrium for a strategic form game (see [20, Chapter 5]), we define a perfect equilibrium for a polymatrix game as follows.
Definition 1. Let be a Nash equilibrium of a polymatrix game with players. The equilibrium is perfect if there exists a sequence of completely mixed strategies -tuples converging to , such that, for all and ,
In other words, a perfect Nash equilibrium is the limit point of a sequence of completely mixed strategy combinations such that, for every player , is a best response against every in every element in this sequence. An equivalent definition which uses -perfect equilibria can also be stated (see [27, Chapter 2]).
Definition 2. Let be a Nash equilibrium of a polymatrix game with players. Given any strictly positive number , with , the equilibrium is perfect if there exists a sequence of -perfect equilibria in completely mixed strategies converging to as goes to zero, such that, for all and ,
This second characterization describes a perfect Nash equilibrium as the limit point of a sequence of -perfect equilibria of the polymatrix game. Every strategy in an -perfect equilibrium is played with a strictly positive probability. As shown by van Damme in his corollary 2.2.6 in [27], the convergence of the sequences of -perfect equilibria to the perfect equilibrium certifies that is undominated. In other words, in every perfect equilibrium, for any given player , any strategy is assigned a zero probability.
3.2. New Definition for Polymatrix Game Perfect Equilibria
In the following, we reformulate the conditions on polymatrix games perfect equilibria to show that every player’s mixed strategic choice is a best response to any combination of the other players pure strategic choices. In other words, we show that every perfect equilibrium of a polymatrix game is undominated and every undominated equilibrium of a polymatrix game is perfect. While this result is known to always be satisfied for bimatrix games, the second part of it is not true in general for games with more than two players. Nevertheless, it appears from the next development that the particular structure of polymatrix games payoffs allows us to extend the perfectness undominance property to polymatrix games. To reach this result, we first show that is a Nash equilibrium of a polymatrix game if and only if .
Proposition 3. The -tuple is a Nash equilibrium of the polymatrix game with players if and only if for each player .
Proof. In the first part of the proof, we show that if is a Nash equilibrium, then for each player . In the second part, we show that if for each player , then is a Nash equilibrium.
Part I ( is a Nash equilibrium). Since is a Nash equilibrium, for each player and for each , we have
Let , with and . Assume that . Then, there exist at least one (), such that and . Thus, we can write . Moreover, there exists at least one strategy , such that . Since , we have
Therefore, .
Hence,
which yields
Since , . Thus, the mixed strategy vector is a strictly better response than , which contradicts the fact that is a Nash equilibrium. Therefore, if is a Nash equilibrium, then for each player .
Part II ( for each player ). Since for each player , , with only if , and . We now refer to by if . Then, for each pair such that (), and , we have
Thus,
If we sum all the , we obtain
Since , . Hence, for each player , is a better response than any mixed strategy vector . We deduce that if for each player , then is a Nash equilibrium.
We conclude that a -tuple is a Nash equilibrium of the polymatrix game with players if and only if , for each player .
As shown in [4], a strategy of a given player is weakly dominated if, for every pure strategic combination of the other players choices, there exists , a convex combination of the pure strategies of player , such that the total payoff for , if he plays this weakly dominated strategy, is always less or equal to his payoff if he plays the convex combination of his pure strategies . In the following development, we show that in every perfect equilibrium, for any given player , any weakly dominated strategy that provides a total payoff strictly less than the total payoff provided by one of the dominant convex combinations of his pure strategies, for some pure strategic combination of the other players choices, should be assigned a zero probability. To do so, let us define to be the set of indices of all pure strategy reply combinations by all players .
Proposition 4. Let be a Nash equilibrium of a polymatrix game. For any player , let be any weakly dominated pure strategy. Also let be a convex combination of the pure strategies of player that weakly dominates . If for some combination the dominant convex combination is such that and if the probability assigned by player to is strictly positive, then is not perfect.
Proof. For any player , if is weakly dominated, there exists a convex combination of all the other pure strategies of player , such that , , and the following inequality is satisfied:
If for some combination , we have , then we can write
By Definition 1, if is a perfect equilibrium, then . Since, for each player , is completely mixed, each pure strategy of in a combination or is assigned a strictly positive probability and , such that .
Thus, we have
and , which is equivalent to
and .
If the weakly dominated strategy is assigned a strictly positive probability , then
and .
Therefore, if we sum, respectively, on all and , we obtain
and , which is equivalent to
Since , adding side by side inequality (24) yields
Also, we know that the mixed strategy vector can be expressed using the weakly dominated strategy and all other pure strategies. Hence, we can write
such that . Thus, if we add the term
to both sides of inequality (25), we obtain
Thus, , which contradicts the fact that is perfect. Therefore, if is a perfect Nash equilibrium, then for any given player , any weakly dominated strategy that provides a total payoff strictly less than the total payoff provided by a dominant convex combination , for some pure strategic combination of the other players choices, should be assigned a zero probability.
We now show that every perfect equilibrium of a polymatrix game is undominated and every undominated equilibrium of a polymatrix game is perfect. Theorem 5 sets an alternate definition of perfect equilibrium for polymatrix games.
Theorem 5. Let be a Nash equilibrium of a polymatrix game, and let be any pure strategy reply vector by any player . The Nash equilibrium is perfect if and only if for each player and any vector , the vector satisfies
Proof. In the first part of the proof, we show that if is a perfect Nash equilibrium, then for each player . In the second part, we show that if for each player , then is a perfect Nash equilibrium.
Part I (the Nash equilibrium is perfect). Firstly, we write , with and . Since is a perfect Nash equilibrium of a polymatrix game, Proposition 4 implies that, for each player , is such that any pure strategy is assigned a nonzero probability only if for any pure strategy reply vector , by any player , we have
Thus, we can write
Therefore, we obtain
Since , . Hence
We deduce that if is a perfect Nash equilibrium, then is a best response to any combination of the other players pure strategic choices.
Part II ( for each player ). Many authors show how to construct a sequence of completely mixed strategies converging to a given Nash equilibrium under some refinement conditions [20, 26]. In the absence of any particular refinement condition, we can assume that it is easy to construct a sequence of completely mixed strategies -tuples converging to .
Let . Then, each real parameter is strictly positive () and . Since for all , for each positive real parameter , we can write
Thus, if we sum all the , we obtain
Hence,
Therefore,
Since , we obtain
which shows that is a best response to every in the sequence
Hence, satisfies the conditions of Definition 1; that is, is a perfect Nash equilibrium.
We finally conclude that the Nash equilibrium is perfect if and only if for any pure strategy reply vector by any player , for each player and any vector , the vector satisfies
This shows that is a perfect Nash equilibrium if and only if it is a best response to any combination of the other players pure strategic choices.
While this perfectness undominance property is generally not right for -player normal form games, Theorem 5 showed how the additive structure of polymatrix games payoffs allows this property to be extended to this particular class of -player games. Hence, if there exists a vector of mixed strategies such that then the equilibrium is not perfect. An immediate corollary can be stated.
Corollary 6. Let be a Nash equilibrium of a polymatrix game. For any player , if there is a vector such that then is not perfect.
This characterization of equilibrium strategies can be used to verify if a Nash equilibrium is perfect or not.
Proposition 7. The equilibrium is perfect if and only if all optimal objective functions values of the following linear programs are equal to zero, for all : where is a column vector with all entries equal to one.
Proof. Let be the optimal solution for a linear program (43), for some . If the optimal objective function value is strictly positive, then at least one of the variables is strictly positive.
In other words, there is at least one , with , such that
Therefore, we have , which means that
while is satisfied. Hence, the equilibrium is not perfect.
If all the optimal objective functions are equal to zero, for all , then all the entries of the vectors are equal to zero. The vectors correspond to the maximum slack vectors between and . Therefore,
Hence, if all the vectors are equal to zero, the equilibrium is perfect.
We note that the linear programs (43) are always feasible for and .
Example 8. Consider a three-player polymatrix game taken from Audet et al. [4], where , , and are the payoff matrices of players I, II, and III, respectively. As presented in Table 1, the MIP algorithm enumerated seven extreme Nash equilibria for this game using exact arithmetics: This game has five maximal Nash subsets , , , , and . For the second extreme Nash equilibrium, the linear program (43), for player III, is expressed as follows:
As in Audet et al. [28], we have used exact arithmetics to obtain exact solutions for these linear programs. For this polymatrix game, all of the seven extreme Nash equilibria enumerated are found to be perfect.
4. Geometry of The Set of Perfect Equilibria
The preceding game example suggests that the enumeration of the extreme Nash equilibria of a polymatrix game leads to a description of the set of Nash perfect equilibria. However, to the best of our knowledge, there are no published results on the geometric properties of the set of Nash perfect equilibria for polymatrix games. By Proposition 9, we show that the set of Nash equilibria of a polymatrix game is a finite union of convex polytopes.
Proposition 9. Let be a polymatrix game with players. Any perfect Nash equilibrium is a convex combination of extreme perfect Nash equilibria.
Proof. Given that is a Nash equilibrium, then can be expressed as a convex combination of a number of extreme Nash equilibria belonging to the same Nash maximal subset .
Let be any extreme Nash equilibrium representing any extreme point of the Nash maximal subset .
Then, we can write , where is the number of extreme Nash equilibria in , , and . Therefore, .
Given that is a perfect equilibrium, then for any vector , the vector is such that
where is a vector of pure strategy reply by any player .
Thus, for any vector , we have
Now let us suppose that can be expressed as a combination of extreme perfect and extreme nonperfect Nash equilibria of the Nash maximal subset . In particular, let be an extreme nonperfect Nash equilibrium of . Also, let be an extreme perfect Nash equilibrium of . Then, we can write
with at least one and at least one .
Thus, for every , . Since all extreme Nash equilibria of are interchangeable and have in common for every , we can write . Hence, , for every . Therefore,
On one hand, condition (52) is satisfied for except for (), since is not perfect. Therefore, there exists a vector such that
Hence, with we have
On the other hand, since is perfect, condition (52) is satisfied for including (). Then, for any vector , we have
Therefore, with we have
Inequalities (54) and (56) imply
Since , we have
It is now made clear that Condition (58) contradicts Condition (50). Therefore, if is a nonperfect extreme Nash equilibrium, then . Hence, any perfect Nash equilibrium is a convex combination of extreme perfect Nash equilibria.
A set of perfect Nash equilibria belonging to the same Nash subset is called a Selten subset. If a Selten subset is not properly contained in another Selten subset, then it is called a maximal Selten subset.
Corollary 10. Any maximal Selten subset is a convex polytope.
Proof. Following Proposition 9, any perfect equilibrium is a convex combination of a number of extreme Nash equilibria belonging to the same maximal Nash subset. Therefore, any maximal Selten subset is a convex polytope.
Example 11. Given the Nash maximal subsets identified for Example 8, the maximal Selten subsets of this game are , , , , and .
Quintas [12] showed that the set of Nash equilibrium points in a polymatrix game is a finite union of convex polytopes. These convex polytopes are possibly disjoint as in Example 8. Following Proposition 9 and Corollary 10, we state that the set of perfect Nash equilibrium points of a polymatrix game is a finite union of convex polytopes, possibly disjoint.
Theorem 12. The set of perfect Nash equilibrium points of a polymatrix game is a finite union of convex polytopes, possibly disjoint.
Proof. Any maximal Selten subset is a convex polytope contained in a maximal Nash subset. Therefore, the set of perfect Nash equilibrium points of a polymatrix game is a finite union of convex polytopes. The maximal Selten subsets are possibly disjoint as in Example 8.
5. Applications
Many applications can be found to illustrate how polymatrix games can be used. In the following, we illustrate our results on a three-player chain store competition game and a three-player inspection management game inspired from Fandel and Trockel [29].
Application 1. Figure 1 illustrates an extensive competition game with imperfect information involving three chain stores. Each of the chain stores 1, 2, and 3 has to decide either to enter the market zones of both of its opponents or not. Hence, each chain store randomizes on two pure strategic decisions “In” and “out.” Each chain store gets a partial payoff depending on its decision and the opponents’ decisions. For example, if chain store 1 decides to get “In” while chain stores 2 and 3 decide to stay out, chain store 1 gets as a total payoff and chain stores 2 and 3 get, respectively, and , respectively. This game can be reduced to a three-person polymatrix game with the following payoff matrices:
Using exact arithmetics, the MIP algorithm enumerated three extreme Nash equilibria for this game, as presented in Table 2. This game has three completely disjoint maximal Nash subsets , , and . The extreme Nash equilibria 1 and 3 are not perfect. In fact, for both of these extreme equilibria, dominates and respectively. The second strategy of player 1 is weakly dominated by his first strategy. Therefore, any Nash equilibrium which assigns to this strategy a strictly positive probability cannot be perfect. The extreme Nash equilibrium 2 is perfect and defines by itself the unique maximal Selten subset of this game. This equilibrium is also the unique subgame perfect Nash equilibrium of the original extensive game with imperfect information.
Application 2. A polymatrix management inspection game involves three players; the manager (M), the controller (C), and the company’s management (U). The manager has to randomize between two pure strategies: , to plan methodically, or , not to plan methodically. The controller controls the manager’s work and has to randomize between two pure strategies: , to compile a precise report on the manager’s activity, or , not to compile a precise report on the manager’s activity. The company’s management inspects the controller’s report and has to randomize between two pure strategies: , to perform an intensive inspection of the controller’s report, or , not to perform an intensive inspection of the controller’s report.
If the manager (M) plans methodically, he is rewarded by the company’s board with a bonus compensation or depending on the company’s board inspection. Otherwise, a penalty or is subtracted from his salary. If the controller (C) compiles a precise report he is rewarded by the company’s board with a bonus amount or depending on the company’s board inspection. Otherwise, a penalty or is subtracted from his remuneration. The manager not planning methodically has a leisure gain payoff . The cost of a precise report to the controller is if the manager plans methodically. Otherwise the cost of the report is . The cost of an intensive inspection to the company’s board is if the controller compiles a precise report. Otherwise the cost is . If the manager plans methodically the company’s board (U) earns a profit . Otherwise a loss is registered. The payoff matrices can be expressed as follows:
The current values of the game parameters are displayed in Table 3.
This game has three extreme Nash equilibria and two maximal Nash subsets and . As presented in Table 4, the extreme Nash equilibria 2 and 3 are not perfect. In fact, for both of these extreme equilibria, dominates . For the manager (D), the weakly dominated strategy cannot be assigned a nonzero probability strategy. The extreme Nash equilibrium 1 is perfect and defines by itself the unique maximal Selten subset of this game.
6. Computational Results
Our computational experiments on randomly generated polymatrix games with different size and density are presented in Table 5. Using C++ implementations of the MIP algorithm, these experimental results were obtained under Windows XP on workstations with 3.3 GHz Intel Core i5 vPro processors and 3.2 GB RAM. The optimization of the linear programs defined by Proposition 7 is performed using an exact arithmetics implementation of the Simplex algorithm [30].
The column (Size) indicates the number of strategies of each player before elimination of strongly dominated strategies as performed in [4]. The value (d) indicates the density of the generated partial payoff matrices. The column (ne) indicates the total number of extreme Nash equilibria enumerated over 10 randomly generated polymatrix games. The column (np) indicates the total number of extreme perfect equilibria obtained. Finally the column (%) indicates the percentage of extreme perfect equilibria.
6.1. Discussion
The experimental results show that the percentage of extreme perfect equilibria increases with an increase of the polymatrix games density. Hence, reducing the partial payoff matrices density increases the number of extreme Nash equilibria of the polymatrix game and decreases the probability of generating perfect extreme equilibria. For three-person polymatrix games, Table 5 illustrates how the percentage of extreme perfect equilibria goes from , for the sparsest case where the polymatrix games density is , to , for a density of . Meanwhile, the average number of extreme Nash equilibria decreases from 53 to 10. This could be explained by the fact that the high number of zeros in the payoff matrices reduces the number of strictly dominated strategies and increases the number of extreme Nash equilibria by offering a number of strategies with the same zero payoff for each player. At the same time, the high number of zeros in the payoff matrices increases the number of weakly dominated strategies which decreases the probability of meeting extreme perfect equilibria. This behavior was already observed for bimatrix games [26].
7. Conclusion
Game theoretic applications have been increasingly encountered in engineering, management and political sciences. Decision makers can often be represented by autonomous agents such as hardware (central units) or software (program applications), which are unable to distinguish between a set of Nash equilibria unless a refinement procedure is used. This paper presents a new characterization of perfect Nash equilibria for polymatrix games. This characterization shows that a Nash equilibrium is perfect if and only if it is a best response to any combination of the other players pure strategic plays. While this characterization is generally not right for -player games as shown by van Damme (see [27, Chapter 2]), the additive structure of polymatrix games payoffs allows the undominance property of perfect equilibria to be extended to this particular class of -player normal form games.
Moreover, we show that any perfect Nash equilibrium is a convex combination of extreme perfect Nash equilibria. As an immediate implication of this result, the set of perfect Nash equilibria of a polymatrix game is a finite union of convex polytopes. A linear programming formulation to identify perfect equilibria for polymatrix games is presented. Finally, the results of this paper are used to perform computational experiments on randomly generated polymatrix games with different size and density.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was partially supported by KFUPM Deanship of research under Grant IN101038. This support is gratefully acknowledged. The author thanks Professors Charles Audet, Bernhard von Stengel, and Issam Louhichi for their constructive and valuable comments on the content of this paper.