- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Volume 2013 (2013), Article ID 534875, 10 pages
Chess-Like Games May Have No Uniform Nash Equilibria Even in Mixed Strategies
RUTCOR, Rutgers University, 640 Bartholomew Road, Piscataway, NJ 08854-8003, USA
Received 2 February 2013; Accepted 22 April 2013
Academic Editor: Walter Briec
Copyright © 2013 Endre Boros et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Recently, it was shown that Chess-like games may have no uniform (subgame perfect) Nash equilibria in pure positional strategies. Moreover, Nash equilibria may fail to exist already in two-person games in which all infinite plays are equivalent and ranked as the worst outcome by both players. In this paper, we extend this negative result further, providing examples that are uniform Nash equilibria free, even in mixed or independently mixed strategies. Additionally, in case of independently mixed strategies we consider two different definitions for effective payoff: the Markovian and the a priori realization.
1.1. Nash-Solvability in Pure and Mixed Strategies: Main Results
There are two very important classes of the so-called uniformly Nash-solvable positional games with perfect information, for which a Nash equilibrium (NE) in pure stationary strategies, which are also independent of the initial position, exists for arbitrary payoffs. These two classes are the two-person zero-sum games and the -person acyclic games.
However, when (directed) cycles are allowed and the game is not zero sum, then a positional game with perfect information may have no uniform NE in pure stationary strategies. This may occur already in the special case of two players with all cycles equivalent and ranked as the worst outcome by both players. Such an example was recently constructed in .
Here we strengthen this result and show that for the same example no uniform NE exists even in mixed stationary strategies, not only in pure ones. Moreover, the same negative result holds for the so-called independently mixed strategies. In the latter case we consider two different definitions for the effective payoffs, based on Markovian and a priori realizations.
In the rest of the introduction we give precise definitions and explain the above result in more details.
Remark 1. In contrast, for the case of a fixed initial position, Nash-solvability in pure positional strategies holds for the two-person case and remains an open problem for ; see  for more details; see also [2–20] for different cases of Nash-solvability in pure strategies.
Furthermore, for a fixed initial position, the solvability in mixed strategies becomes trivial, due to the general result of Nash [21, 22]. Thus, our main example shows that Nash’s theorem cannot be extended for positional games to the case of uniform equilibria. It is shown for the following four types of positional strategies: pure, mixed, and independently mixed, where in the last case we consider two types of effective payoffs, defined by Markovian and a priori realizations.
1.2. Positional Game Structures
Given a finite directed graph (digraph) in which loops and multiple arcs are allowed, a vertex is interpreted as a position and a directed edge (arc) as a move from to . A position of outdegree (one with no moves) is called terminal. Let be the set of all terminal positions. Let us also introduce a set of players and a partition , assuming that each player is in control of all positions in .
An initial position may be fixed. The triplet or pair is called a Chess-like positional game structure (or just a game structure, for short), initialized or noninitialized, respectively. By default, we assume that it is not initialized.
Two examples of (noninitialized) game structures and are given in Figure 1.
1.3. Plays, Outcomes, Preferences, and Payoffs
Given an initialized positional game structure , a play is defined as a directed path that begins in and either ends in a terminal position or is infinite. In this paper we assume that all infinite plays form one outcome (or ), in addition to the standard Terminal outcomes of . (In , this condition was referred to as AIPFOOT.)
A utility (or payoff) function is a mapping , whose value is interpreted as a profit of the player in case of the outcome .
A payoff is called zero-sum if for every . Two-person zero sum games are important. For example, the standard Chess and Backgammon are two-person zero-sum games in which every infinite play is a draw, . It is easy to realize that can be assumed for all players without any loss of generality.
Another important class of payoffs is defined by the condition for all and ; in other words, the infinite outcome is ranked as the worst one by all players. Several possible motivations for this assumption are discussed in [4, 5].
A quadruple and triplet will be called a Chess-like game, initialized and noninitialized, respectively.
Remark 2. From the other side, the Chess-like games can be viewed as the transition-free deterministic stochastic games with perfect information; see, for example, [8–11].
In these games, every nonterminal position is controlled by a player and the local reward is for each player and move , unless is a terminal move, that is, . Obviously, in the considered case all infinite plays are equivalent since the effective payoff is for every such play. Furthermore, obviously, is the worst outcome for a player if and only if for every terminal move .
If , then the zero-sum Chess-like games turn into a subclass of the so-called simple stochastic games, which were introduced by Condon in .
1.4. Pure Positional Strategies
Given game structure , a (pure positional) strategy of a player is a mapping that assigns to each position a move from this position.
The concept of mixed strategies will be considered in Section 1.10; till then only pure strategies are considered. Moreover, in this paper, we restrict the players to their positional (pure) strategies. In other words, the move of a player in a position depends only on the position itself, not on the preceding positions or moves.
Let be the set of all strategies of a player and be the direct product of these sets. An element is called a strategy profile or situation.
1.5. Normal Forms
A positional game structure can be represented in the normal (or strategic) form.
Let us begin with the initialized case. Given a game structure and a strategy profile , a play is uniquely defined by the following rules: it begins in and in each position proceeds with the arc determined by the strategy . Obviously, either ends in a terminal position or is infinite. In the latter case is a lasso; that is, it consists of an initial part and a directed cycle (dicycle) repeated infinitely. This holds, because all players are restricted to their positional strategies. In either case, an outcome is assigned to each strategy profile . Thus, a game form is defined. It is called the normal form of the initialized positional game structure .
If the game structure is not initialized, then we repeat the above construction for every initial position to obtain a play , outcome , and mapping , which is the normal form of in this case. In general we have . For the (noninitialized) game structures in Figure 1 their normal forms are given in Figures 2 and 3.
Given also a payoff , the pairs and define the games in the normal form, for the above two cases.
Of course, these games can be also represented by the corresponding real-valued mappings: where for all , , .
Remark 3. Yet, it seems convenient to separate the game from and utility function .
By this approach, “takes responsibility for structural properties” of the game , that is, the properties that hold for any .
1.6. Nash Equilibria in Pure Strategies
First, let us consider the initialized case. Given and , a situation is called a Nash equilibrium (NE) in the normal form game if for each player and every strategy profile that can differ from only in the th component. In other words, no player can profit by choosing a new strategy if all opponents keep their old strategies.
In the noninitialized case, the similar property is required for each . Given a payoff , a strategy profile is called a uniform NE if for each , every defined as above, and for all , too.
Remark 4. In the literature, the last concept is frequently called a subgame perfect NE rather than a uniform NE. This name is justified when the digraph is acyclic and each vertex can be reached from . Indeed, in this case is a subgame of for each . However, if has a dicycle then any two its vertices and can be reached one from the other; that is, is a subgame of and vice versa. Thus, the name uniform (or ergodic) NE seems more accurate.
1.7. Uniformly Best Responses
Again, let us start with the initialized case. Given the normal form of an initialized Chess-like game, a player , and a pair of strategy profiles such that may differ from only in the th component, we say that improves (for the player ) if . Let us underline that the inequality is strict. Furthermore, by this definition, a situation is a NE if and only if it can be improved by no player ; in other words, any sequence of improvements either can be extended, or terminates in an NE.
Given a player and situation , a strategy is called a best response (BR) of in if for any , where and are both obtained from by replacement of its th component by and , respectively. A BR is not necessarily unique but the corresponding best achievable value is, of course, unique. Moreover, somewhat surprisingly, such best values can be achieved by a BR simultaneously for all initial positions . (See, e.g., [1, 4–6], of course, this result is well known in much more general probabilistic setting; see, e.g., textbooks [24–26].)
Theorem 5. Let be the normal form of a (noninitialized) Chess-like game . Given a player and a situation , there is a (pure positional) strategy which is a BR of in for all initial positions simultaneously.
We will call such a strategy a uniformly BR of the player in the situation . Obviously, the nonstrict inequality holds for each position . We will say that improves if this inequality is strict, , for at least one . This statement will serve as the definition of a uniform improvement for the noninitialized case. Let us remark that, by this definition, a situation is a uniform NE if and only if can be uniformly improved by no player ; in other words, any sequence of uniform improvements either can be extended or terminates in a uniform NE.
Given a noninitialized Chess-like game , a player , and a strategy profile , in every position let us fix a move in accordance with and delete all other moves. Then, let us order according to the preference . Let be a best outcome. (Note that there might be several such outcomes and also that might hold.) Let denote the set of positions from which player can reach (in particular, ). Let us fix corresponding moves in . Obviously, there is no move to from . Moreover, if , then player cannot reach a dicycle beginning from ; in particular, the induced digraph contains no dicycle.
Then, let us consider an outcome that is the best for in , except maybe , and repeat the same arguments as above for and , and so forth. This procedure will result in a uniformly BR of in since the chosen moves of are optimal independently of .
1.8. Two Open Problems Related to Nash-Solvability of Initialized Chess-Like Game Structures
Given an initialized game structure , it is an open question whether an NE (in pure positional strategies) exists for every utility function . In , the problem was raised and solved in the affirmative for two special cases: or . The last result was strengthened to in . More details can be found in  and in the last section of .
In general the above problem is still open even if we assume that is the worst outcome for all players.
Yet, if we additionally assume that is play-once (i.e., for each ), then the answer is positive . However, in the next subsection we will show that it becomes negative if we ask for the existence of a uniform NE rather than an initialized one.
1.9. Chess-Like Games with a Unique Dicycle and without Uniform Nash Equilibria in Pure Positional Strategies
Let us consider two noninitialized Chess-like positional game structures and given in Figure 1. For , the corresponding digraph consists of a unique dicycle of length and a matching connecting each vertex of to a terminal , where and . The digraph is bipartite; respectively, is a two-person game structures in which two players take turns; in other words, players and control positions and , respectively. In contrast, is a play-once three-person game structure, that is, each player controls a unique position. In every nonterminal position there are only two moves: one of them () immediately terminates in , while the other one () proceeds to ; by convention, we assume .
Remark 6. In Figure 1, the symbols for the terminal positions are shown but for the corresponding positions of the dicycle are skipped; moreover, in Figures 1–3, we omit the superscript in , for simplicity and to save space.
Thus, in each player has two strategies coded by the letters and , while in each player has strategies coded by the -letter words in the alphabet . For example, the strategy of player in requires to proceed to from and to terminate in from and in from .
The corresponding normal game forms and of size and are shown in Figures 2 and 3, respectively. Since both game structures are noninitialized, each situation is a set of and terminals, respectively. These terminals correspond to the nonterminal positions of and , each of which can serve as an initial position.
In other words, for each player to terminate is an average outcome; it is better (worse) when the next (previous) player terminates; finally, if nobody does, then the dicycle appears, which is the worst outcome for all. The considered game has an improvement cycle of length , which is shown in Figure 2. Indeed, let player terminates at , while and proceed. The corresponding situation can be improved by to , which in its turn can be improved by to . Repeating the similar procedure two times more we obtain the improvement cycle shown in Figure 2.
There are two more situations, which result in and . They appear when all three players terminate or proceed simultaneously. Yet, none of these two situations is an NE either. Moreover, each of them can be improved by every player .
Theorem 7. Game has no uniform NE in pure strategies whenever .
We note that each player has positive payoffs. This is without loss of generality as we can shift the payoffs by a positive constant without changing the game.
A similar two-person uniform NE-free example was suggested in , for . Let us consider a family of the utility functions defined by the following constraints:
We claim that the Chess-like game has no uniform NE whenever .
Let us remark that and that is the worst outcome for both players for all . To verify this, let us consider the normal form in Figure 3. By Theorem 5, there is a uniformly BR of player to each strategy of player and vice versa. It is not difficult to check that the obtained two sets of the BRs (which are denoted by the white discs and black squares in Figure 3) are disjoint. Hence, there is no uniform NE. Furthermore, it is not difficult to verify that the obtained situations induce an improvement cycle of length and two improvement paths of lengths and that end in this cycle.
Theorem 8 (see ). Game has no uniform NE in pure strategies whenever .
The goal of the present paper is to demonstrate that the above two game structures may have no uniform NE not only in pure but also in mixed strategies. Let us note that by Nash’s theorem [21, 22] NE in mixed strategies exist in any initialized game structure. Yet, this result cannot be extended to the noninitialized game structure and uniform NE. In this research we are motivated by the results of [8, 11].
1.10. Mixed and Independently Mixed Strategies
Standardly, a mixed strategy of a player is defined as a probabilistic distribution over the set of his pure strategies. Furthermore, is called an independently mixed strategy if randomizes in his positions independently. We will denote by and by the sets of mixed and independently mixed strategies of player , respectively.
Remark 9. Let us recall that the players are restricted to their positional strategies and let us also note that the latter concept is closely related to the so-called behavioral strategies introduced by Kuhn [19, 20]. Although Kuhn restricted himself to trees, yet his construction can be extended to directed graphs, too.
Let us recall that a game structure is called play-once if each player is in control of a unique position. For example, is play-once. Obviously, the classes of mixed and independently mixed strategies coincide for a play-once game structure. However, for these two notion differ. Each player controls positions and has pure strategies. Hence, the set of mixed strategies is of dimension , while the set of the independently mixed strategies is only -dimensional.
2. Markovian and A Priori Realizations
For the independently mixed strategies we will consider two different options.
For every player let us consider a probability distribution for all positions , which assigns a probability to each move from , standardly assuming
Now, the limit distributions of the terminals can be defined in two ways, which we will be referred to as the Markovian and a priori realizations.
For example, let us consider and let be the probability to proceed in for . If , then, obviously, the play will cycle with probability resulting in the limit distribution for . Otherwise, assuming that is the initial position, we obtain the limit distribution:
Indeed, positions are transient and the probability of cycling forever is whenever . Obviously, the sum of the above four probabilities is .
The Markovian approach assumes that for the move is chosen randomly, in accordance with the distribution , and independently for all (furthermore, is a fixed initial position). In particular, if the play comes to the same position again, that is, for some , then the moves and may be distinct although they are chosen (independently) with the same distribution .
The concept of a priori realization is based on the following alternative assumptions. A move is chosen according to , independently for all , but only once, before the game starts. Being chosen the move is applied whenever the play comes at . By these assumptions, each infinite play is a lasso; that is, it consists of an initial part (that might be empty) and an infinitely repeated dicycle . Alternatively, may be finite; that is, it terminates in a . In both cases, begins in and the probability of is the product of the probabilities of all its moves, . In this way, we obtain a probability distribution on the set of lassos of the digraph. In particular, the effective payoff is defined as the expected payoffs for the corresponding lassos. Let us also note that (in contrast to the Markovian case) the computation of limit distribution is not computationally efficient, since the set of plays may grow exponentially in size of the digraph. No polynomial algorithm computing the limit distribution is known for a priori realizations. Returning to our example , we obtain the following limit distribution: with initial position . The probability of outcome is ; it is strictly positive whenever for all . Indeed, in contrast to the Markovian realization, the cycle will be repeated infinitely whenever it appears once under a priori realization.
Remark 10. Thus, solving the Chess-like games in the independently mixed strategies looks more natural under a priori (rather than Markovian) realizations. Unfortunately, it seems not that easy to suggest more applications of a priori realizations and we have to acknowledge that the concept of the Markovian realization is much more fruitful. Let us also note that playing in pure strategies can be viewed as a special case of both Markovian and a priori realizations with degenerate probability distributions.
As we already mentioned, the mixed and independently mixed strategies coincide for since it is play-once. Yet, these two classes of strategies differ in .
3. Chess-Like Games with No Uniform NE
In the present paper, we will strengthen Theorems 7 and 8, showing that games and may fail to have an NE (not only in pure, but even) in mixed strategies, as well as in the independently mixed strategies, under both Markovian and a priori realizations.
For convenience, let denote the set of indices of nonterminal positions. We will refer to positions giving only these indices.
Let us recall the definition of payoff function of player for the initial position and the strategy profile ; see Theorem 5. Let us extend this definition introducing the payoff function for the mixed and independently mixed strategies. In both cases, we define it as the expected payoff, under one of the above realizations, and denote by , where is an -vector whose th coordinate is the probability of proceeding (not terminating) at position .
Remark 11. Let us observe that, in both and , the payoff functions , are continuously differentiable functions of when for all , for all players . Hence, if is a uniform NE such that for all (under either a priori or Markovian realization), then
In the next two sections, we will construct games that have no uniform NE under both, a priori and Markovian, realizations. Assuming that a uniform mixed NE exists, we will obtain a contradiction with (7) whenever for all .
The next lemma will be instrumental in the proofs of the following two theorems.
Lemma 12. The probabilities to proceed satisfy for all in any independently mixed uniform NE in game , where , and under both a priori and Markovian realizations.
Proof. Let us assume indirectly that there is an (independently) mixed uniform NE under a priori realization with for some . This would imply the existence of an acyclic game with uniform NE, in contradiction with Theorem 7. Now let us consider the case . Due to the circular symmetry of , we can choose any player, say, . The preference list of player is . His most favorable outcome, , is not achievable since . Hence, because his second best outcome is . Thus, the game is reduced to an acyclic one, in contradiction with Theorem 7, again.
Theorem 13. Game has no uniform NE in independently mixed strategies under a priori realization whenever .
Proof. To simplify our notation we denote by and the following and preceding positions along the 3-cycle of , respectively. Assume indirectly that forms a uniform NE and considers the effective payoff of player 1:
where is the initial position.
By Lemma 12, we must have for . Therefore (7) must hold. Hence, and follows since . Thus, , in contradiction to our assumption.
Let us recall that for , independently mixed strategies and mixed strategies are the same.
Now, let us consider the Markovian realization. Game may have no NE in mixed strategies under Markovian realization either, yet, only for some special payoffs .
Theorem 14. Game , with , has no uniform NE in independently mixed strategies under Markovian realization if and only if , where
It is easy to verify that for whenever . Let us also note that in the symmetric case the above condition turns into .
Proof. Let be a uniform NE in the game under Markovian realization. Then, by Lemma 12, for . The payoff function of a player, with respect to the initial position that this player controls, is given by one of the next three formulas:
By Lemma 12, (7) holds for any uniform NE. Therefore we have
Setting for , we can transform the above equations to the following form: Assuming , and using successive elimination, we uniquely express via as follows:
Interestingly, all three inequalities are equivalent with the condition , that is, , which completes the proof.
Here we will show that may have no uniform NE for both Markovian and a priori realizations, in independently mixed strategies, whenever . As for the mixed (unlike the independently mixed) strategies, we obtain NE-free examples only for some (not for all) .
We begin with extending Lemma 12 to game and as follows.
Lemma 15. The probabilities to proceed satisfy for all in any independently mixed uniform NE in game , where , and under both a priori and Markovian realizations.
Proof. To prove that for all let us consider the following six cases: (i)If , then player will proceed at position , as in , implying . (ii)If , then either or , as player 1, prefers to . (iii)If , then , as player cannot achieve his best outcome of , while is his second best one. (iv)If , then , as player 1’s worst outcome is in the current situation. (v)If , then , as player , prefers to . (vi)If , then , as player 1’s best outcome, is now. It is easy to verify that, by the above implications, in all six cases at least one of the proceeding probabilities should be , in contradiction to Theorem 8.
Let us show that the game might have no NE in independently mixed strategies under both Markovian and a priori realizations. Let us consider the Markovian one first.
Theorem 16. Game has no uniform NE in the independently mixed strategies under Markovian realization for all .
Proof. Let us consider the uniform NE conditions for player 2. Lemma 15 implies that (7) must be satisfied. Applying it to the partial derivatives with respect to and we obtain
Let us multiply the first equation by and subtract it from the second one, yielding or equivalently, . From this equation, we find Furthermore, the condition implies that either or . Both orders contradict the preference list , thus, completing the proof.
Now let us consider the case of a priori realization.
Theorem 17. Game has no uniform NE in independently mixed strategies under a priori realization for all .
Proof. Let us assume indirectly that form a uniform NE. Let us consider the effective payoff of the player 1 with respect to the initial position : By Lemma 15, we have for . Hence, (7) must hold; in particular, and, since is positive, we obtain , that is a contradiction.
The last result can be extended from the independently mixed to mixed strategies. However, the corresponding example is constructed not for all but only for some .
Theorem 18. The game has no uniform, NE in mixed strategies, at least for some .
Proof. Let us recall that there are two players in controling three positions each and there are two possible moves in every position. Thus, each player has eight pure strategies. Standardly, the mixed strategies are defined as probability distributions on the set of the pure strategies, that is, , where if and only if and .
Furthermore, let us denote by the outcome of the game beginning in the initial position in case when player 1 chooses his pure strategy and player chooses her pure strategy , where .
Given a utility function , if a pair of mixed strategies form a uniform NE then must hold for some value for all initial positions . Indeed, otherwise player would change the probability distribution to get a better value. Let denote the set of indices of all positive components of . By (19), there exists a subset such that the next system is feasible:
Then, let us consider, for example, a utility function with the following payoffs of player 2: We verified that (19) is infeasible for all subsets such that . Since for any there is no pure strategy NE either, we obtain a contradiction.
3.3. Concluding Remarks
Remark 19. In the last two theorems, in contrast with Theorem 14, uniform NE exist for no .
Remark 20. Let us note that Nash’s results [21, 22], guaranteeing the existence of an NE in mixed strategies for any normal form games, are applicable in case of a fixed initial position. Yet, our results show that Nash’s theorem, in general, does not extend to the case of uniform NE, except for the -person acyclic case [12, 19, 20] and the two-person zero sum cases.
Remark 21. It seems that the same holds for all . We tested (19) for many randomly chosen and encountered infeasibility for all such that . Yet, we have no proof and it still remains open whether for any there is no NE in mixed strategies.
Remark 22. Finally, let us note that for an arbitrary Chess-like game structure (not only for and ) in independently mixed strategies under both the Markovian and a priori realizations for any and , the ratio is a positive constant.
The first and third authors acknowledge the partial support by the NSF Grants IIS-1161476 and also CMMI-0856663. The second author is thankful to János Flesch for helpful discussions. All author are also thankful to an anonymous reviewer for many helpful remarks and suggestions.
- E. Boros, K. Elbassioni, V. Gurvich, and K. Makino, “On Nash equilibria and improvement cycles in pure positional strategies for Chess-like and Backgammon-like n-person games,” Discrete Mathematics, vol. 312, no. 4, pp. 772–788, 2012.
- D. Andersson, V. Gurvich, and T. D. Hansen, “On acyclicity of games with cycles,” Discrete Applied Mathematics, vol. 158, no. 10, pp. 1049–1063, 2010.
- D. Andersson, V. Gurvich, and T. D. Hansen, “On acyclicity of games with cycles,” in Algorithmic Aspects in Information and Management, vol. 5564, pp. 15–28, 2009.
- E. Boros and V. Gurvich, “On Nash-solvability in pure stationary strategies of finite games with perfect information which may have cycles,” Mathematical Social Sciences, vol. 46, no. 2, pp. 207–241, 2003.
- E. Boros and V. Gurvich, “Why chess and backgammon can be solved in pure positional uniformly optimal strategies,” RUTCOR Research Report 21-2009, Rutgers University.
- E. Boros, V. Gurvich, K. Makino, and W. Shao, “Nash-solvable two-person symmetric cycle game forms,” Discrete Applied Mathematics, vol. 159, no. 15, pp. 1461–1487, 2011.
- E. Boros and R. Rand, “Terminal games with three terminals have proper Nash equilibria,” RUTCOR Research Report RRR-22-2009, Rutgers University.
- J. Flesch, J. Kuipers, G. Shoenmakers, and O. J. Vrieze, “Subgame-perfect equilibria in free transition games,” Research Memorandum RM/08/027, University of Maastricht, Maastricht, The Netherlands, 2008.
- J. Flesch, J. Kuipers, G. Shoenmakers, and O. J. Vrieze, “Subgame-perfection equilibria in stochastic games with perfect information and recursive payos,” Research Memorandum RM/08/041, University of Maastricht, Maastricht, The Netherlands, 2008.
- J. Kuipers, J. Flesch, G. Schoenmakers, and K. Vrieze, “Pure subgame-perfect equilibria in free transition games,” European Journal of Operational Research, vol. 199, no. 2, pp. 442–447, 2009.
- J. Flesch, J. Kuipers, G. Schoenmakers, and K. Vrieze, “Subgame perfection in positive recursive games with perfect information,” Mathematics of Operations Research, vol. 35, no. 1, pp. 193–207, 2010.
- D. Gale, “A theory of N-person games with perfect information,” Proceedings of the National Academy of Sciences, vol. 39, no. 6, pp. 496–501, 1953.
- V. A. Gurvich, “On theory of multistep games,” USSR Computational Mathematics and Mathematical Physics, vol. 13, no. 6, pp. 143–161, 1973.
- V. A. Gurvich, “The solvability of positional games in pure strategies,” USSR Computational Mathematics and Mathematical Physics, vol. 15, no. 2, pp. 74–87, 1975.
- V. Gurvich, “Equilibrium in pure strategies,” Soviet Mathematics, vol. 38, no. 3, pp. 597–602, 1989.
- V. Gurvich, “A stochastic game with complete information and without equilibrium situations in pure stationary strategies,” Russian Mathematical Surveys, vol. 43, no. 2, pp. 171–172, 1988.
- V. Gurvich, “A theorem on the existence of equilibrium situations in pure stationary strategies for ergodic extensions of () bimatrix games,” Russian Mathematical Surveys, vol. 45, no. 4, pp. 170–172, 1990.
- V. Gurvich, “Saddle point in pure strategies,” Russian Academy of Science Doklady Mathematics, vol. 42, no. 2, pp. 497–501, 1990.
- H. Kuhn, “Extensive games,” Proceedings of the National Academy of Sciences, vol. 36, pp. 286–295, 1950.
- H. Kuhn, “Extensive games and the problems of information,” Annals of Mathematics Studies, vol. 28, pp. 193–216, 1953.
- J. Nash, “Equilibrium points in n-person games,” Proceedings of the National Academy of Sciences, vol. 36, no. 1, pp. 48–49, 1950.
- J. Nash, “Non-cooperative games,” Annals of Mathematics, vol. 54, no. 2, pp. 286–295, 1951.
- A. Condon, “An algorithm for simple stochastic games,” in Advances in Computational Complexity Theory, vol. 13 of DIMACS series in discrete mathematics and theoretical computer science, 1993.
- I. V. Romanovsky, “On the solvability of Bellman's functional equation for a Markovian decision process,” Journal of Mathematical Analysis and Applications, vol. 42, no. 2, pp. 485–498, 1973.
- R. A. Howard, Dynamic Programming and Markov Processes, The M.I.T. Press, 1960.
- H. Mine and S. Osaki, Markovian Decision Process, American Elsevier, New York, NY, USA, 1970.
- J. G. Kemeny and J. L. Snell, Finite Markov Chains, Springer, 1960.