Abstract

Recently, it was shown that Chess-like games may have no uniform (subgame perfect) Nash equilibria in pure positional strategies. Moreover, Nash equilibria may fail to exist already in two-person games in which all infinite plays are equivalent and ranked as the worst outcome by both players. In this paper, we extend this negative result further, providing examples that are uniform Nash equilibria free, even in mixed or independently mixed strategies. Additionally, in case of independently mixed strategies we consider two different definitions for effective payoff: the Markovian and the a priori realization.

1. Introduction

1.1. Nash-Solvability in Pure and Mixed Strategies: Main Results

There are two very important classes of the so-called uniformly Nash-solvable positional games with perfect information, for which a Nash equilibrium (NE) in pure stationary strategies, which are also independent of the initial position, exists for arbitrary payoffs. These two classes are the two-person zero-sum games and the -person acyclic games.

However, when (directed) cycles are allowed and the game is not zero sum, then a positional game with perfect information may have no uniform NE in pure stationary strategies. This may occur already in the special case of two players with all cycles equivalent and ranked as the worst outcome by both players. Such an example was recently constructed in [1].

Here we strengthen this result and show that for the same example no uniform NE exists even in mixed stationary strategies, not only in pure ones. Moreover, the same negative result holds for the so-called independently mixed strategies. In the latter case we consider two different definitions for the effective payoffs, based on Markovian and a priori realizations.

In the rest of the introduction we give precise definitions and explain the above result in more details.

Remark 1. In contrast, for the case of a fixed initial position, Nash-solvability in pure positional strategies holds for the two-person case and remains an open problem for ; see [1] for more details; see also [2–20] for different cases of Nash-solvability in pure strategies.

Furthermore, for a fixed initial position, the solvability in mixed strategies becomes trivial, due to the general result of Nash [21, 22]. Thus, our main example shows that Nash’s theorem cannot be extended for positional games to the case of uniform equilibria. It is shown for the following four types of positional strategies: pure, mixed, and independently mixed, where in the last case we consider two types of effective payoffs, defined by Markovian and a priori realizations.

1.2. Positional Game Structures

Given a finite directed graph (digraph) in which loops and multiple arcs are allowed, a vertex is interpreted as a position and a directed edge (arc) as a move from to . A position of outdegree (one with no moves) is called terminal. Let be the set of all terminal positions. Let us also introduce a set of players and a partition , assuming that each player is in control of all positions in .

An initial position may be fixed. The triplet or pair is called a Chess-like positional game structure (or just a game structure, for short), initialized or noninitialized, respectively. By default, we assume that it is not initialized.

Two examples of (noninitialized) game structures and are given in Figure 1.

Figure 1 

Two Chess-like game structures and . In , there are players controlling one position each (i.e., is play-once), while in there are only two players who alternate turns; hence, each of them controls positions: are controlled by player and by player . In each position , the corresponding player has only two options: () to proceed to and () to terminate at , where () in and () in . To save space, we show only symbols , while the corresponding vertex names are omitted.

1.3. Plays, Outcomes, Preferences, and Payoffs

Given an initialized positional game structure , a play is defined as a directed path that begins in and either ends in a terminal position or is infinite. In this paper we assume that all infinite plays form one outcome (or ), in addition to the standard Terminal outcomes of . (In [5], this condition was referred to as AIPFOOT.)

A utility (or payoff) function is a mapping , whose value is interpreted as a profit of the player in case of the outcome .

A payoff is called zero-sum if for every . Two-person zero sum games are important. For example, the standard Chess and Backgammon are two-person zero-sum games in which every infinite play is a draw, . It is easy to realize that can be assumed for all players without any loss of generality.

Another important class of payoffs is defined by the condition for all and ; in other words, the infinite outcome is ranked as the worst one by all players. Several possible motivations for this assumption are discussed in [4, 5].

A quadruple and triplet will be called a Chess-like game, initialized and noninitialized, respectively.

Remark 2. From the other side, the Chess-like games can be viewed as the transition-free deterministic stochastic games with perfect information; see, for example, [8–11].
In these games, every nonterminal position is controlled by a player and the local reward is for each player and move , unless is a terminal move, that is, . Obviously, in the considered case all infinite plays are equivalent since the effective payoff is for every such play. Furthermore, obviously, is the worst outcome for a player if and only if for every terminal move .
If , then the zero-sum Chess-like games turn into a subclass of the so-called simple stochastic games, which were introduced by Condon in [23].

1.4. Pure Positional Strategies

Given game structure , a (pure positional) strategy of a player is a mapping that assigns to each position a move from this position.

The concept of mixed strategies will be considered in Section 1.10; till then only pure strategies are considered. Moreover, in this paper, we restrict the players to their positional (pure) strategies. In other words, the move of a player in a position depends only on the position itself, not on the preceding positions or moves.

Let be the set of all strategies of a player and be the direct product of these sets. An element is called a strategy profile or situation.

1.5. Normal Forms

A positional game structure can be represented in the normal (or strategic) form.

Let us begin with the initialized case. Given a game structure and a strategy profile , a play is uniquely defined by the following rules: it begins in and in each position proceeds with the arc determined by the strategy . Obviously, either ends in a terminal position or is infinite. In the latter case is a lasso; that is, it consists of an initial part and a directed cycle (dicycle) repeated infinitely. This holds, because all players are restricted to their positional strategies. In either case, an outcome is assigned to each strategy profile . Thus, a game form is defined. It is called the normal form of the initialized positional game structure .

If the game structure is not initialized, then we repeat the above construction for every initial position to obtain a play , outcome , and mapping , which is the normal form of in this case. In general we have . For the (noninitialized) game structures in Figure 1 their normal forms are given in Figures 2 and 3.

Figure 2 

The normal form of the positional game structures from Figure 1. Each player has only two strategies: to terminate () or proceed (). Hence, is represented by a table, each entry of which contains terminals corresponding to the potential initial positions of . The rows and columns are the strategies of the players and , while two strategies of the player are the left and right subtables. The corresponding game has no uniform NE whenever a utility function satisfies the constraints specified in the figure.

Figure 3 

The normal form of the positional game structures from Figure 1. There are two players controlling positions each. Again, in every position there are only two options: to terminate () or proceed (). Hence, in , each player has strategies, which are naturally coded by the -letter words in the alphabet . Respectively, is represented by the table, each entry of which contains terminals corresponding to the (nonterminal) potential initial positions of . Again, players and control the rows and columns, respectively. The corresponding game has no uniform NE whenever a utility function satisfies the constraints specified under the table. Indeed, a (unique) uniformly best response of the player (resp. ) to each strategy of (resp. ) is shown by the white discs (resp. black squares). Since the obtained two sets are disjoint, no uniform NE exists in .

Given also a payoff , the pairs and define the games in the normal form, for the above two cases.

Of course, these games can be also represented by the corresponding real-valued mappings: where for all , , .

Remark 3. Yet, it seems convenient to separate the game from and utility function .
By this approach, “takes responsibility for structural properties” of the game , that is, the properties that hold for any .

1.6. Nash Equilibria in Pure Strategies

The concept of Nash equilibria is defined standardly [21, 22] for the normal form games.

First, let us consider the initialized case. Given and , a situation is called a Nash equilibrium (NE) in the normal form game if for each player and every strategy profile that can differ from only in the th component. In other words, no player can profit by choosing a new strategy if all opponents keep their old strategies.

In the noninitialized case, the similar property is required for each . Given a payoff , a strategy profile is called a uniform NE if for each , every defined as above, and for all , too.

Remark 4. In the literature, the last concept is frequently called a subgame perfect NE rather than a uniform NE. This name is justified when the digraph is acyclic and each vertex can be reached from . Indeed, in this case is a subgame of for each . However, if has a dicycle then any two its vertices and can be reached one from the other; that is, is a subgame of and vice versa. Thus, the name uniform (or ergodic) NE seems more accurate.

1.7. Uniformly Best Responses

Again, let us start with the initialized case. Given the normal form of an initialized Chess-like game, a player , and a pair of strategy profiles such that may differ from only in the th component, we say that improves (for the player ) if . Let us underline that the inequality is strict. Furthermore, by this definition, a situation is a NE if and only if it can be improved by no player ; in other words, any sequence of improvements either can be extended, or terminates in an NE.

Given a player and situation , a strategy is called a best response (BR) of in if for any , where and are both obtained from by replacement of its th component by and , respectively. A BR is not necessarily unique but the corresponding best achievable value is, of course, unique. Moreover, somewhat surprisingly, such best values can be achieved by a BR simultaneously for all initial positions . (See, e.g., [1, 4–6], of course, this result is well known in much more general probabilistic setting; see, e.g., textbooks [24–26].)

Theorem 5. Let be the normal form of a (noninitialized) Chess-like game . Given a player and a situation , there is a (pure positional) strategy which is a BR of in for all initial positions simultaneously.

We will call such a strategy a uniformly BR of the player in the situation . Obviously, the nonstrict inequality holds for each position . We will say that improves if this inequality is strict, , for at least one . This statement will serve as the definition of a uniform improvement for the noninitialized case. Let us remark that, by this definition, a situation is a uniform NE if and only if can be uniformly improved by no player ; in other words, any sequence of uniform improvements either can be extended or terminates in a uniform NE.

For completeness, let us repeat here the simple proof of Theorem 5 suggested in [1].

Given a noninitialized Chess-like game , a player , and a strategy profile , in every position let us fix a move in accordance with and delete all other moves. Then, let us order according to the preference . Let be a best outcome. (Note that there might be several such outcomes and also that might hold.) Let denote the set of positions from which player can reach (in particular, ). Let us fix corresponding moves in . Obviously, there is no move to from . Moreover, if , then player cannot reach a dicycle beginning from ; in particular, the induced digraph contains no dicycle.

Then, let us consider an outcome that is the best for in , except maybe , and repeat the same arguments as above for and , and so forth. This procedure will result in a uniformly BR of in since the chosen moves of are optimal independently of .

1.8. Two Open Problems Related to Nash-Solvability of Initialized Chess-Like Game Structures

Given an initialized game structure , it is an open question whether an NE (in pure positional strategies) exists for every utility function . In [4], the problem was raised and solved in the affirmative for two special cases: or . The last result was strengthened to in [7]. More details can be found in [1] and in the last section of [6].

In general the above problem is still open even if we assume that is the worst outcome for all players.

Yet, if we additionally assume that is play-once (i.e., for each ), then the answer is positive [4]. However, in the next subsection we will show that it becomes negative if we ask for the existence of a uniform NE rather than an initialized one.

1.9. Chess-Like Games with a Unique Dicycle and without Uniform Nash Equilibria in Pure Positional Strategies

Let us consider two noninitialized Chess-like positional game structures and given in Figure 1. For , the corresponding digraph consists of a unique dicycle of length and a matching connecting each vertex of to a terminal , where and . The digraph is bipartite; respectively, is a two-person game structures in which two players take turns; in other words, players and control positions and , respectively. In contrast, is a play-once three-person game structure, that is, each player controls a unique position. In every nonterminal position there are only two moves: one of them () immediately terminates in , while the other one () proceeds to ; by convention, we assume .

Remark 6. In Figure 1, the symbols for the terminal positions are shown but for the corresponding positions of the dicycle are skipped; moreover, in Figures 1–3, we omit the superscript in , for simplicity and to save space.

Thus, in each player has two strategies coded by the letters and , while in each player has strategies coded by the -letter words in the alphabet . For example, the strategy of player in requires to proceed to from and to terminate in from and in from .

The corresponding normal game forms and of size and are shown in Figures 2 and 3, respectively. Since both game structures are noninitialized, each situation is a set of and terminals, respectively. These terminals correspond to the nonterminal positions of and , each of which can serve as an initial position.

A uniform NE free example for was suggested in [4]; see also [1, 8]. Let us consider a family of the utility functions defined by the following constraints:

In other words, for each player to terminate is an average outcome; it is better (worse) when the next (previous) player terminates; finally, if nobody does, then the dicycle appears, which is the worst outcome for all. The considered game has an improvement cycle of length , which is shown in Figure 2. Indeed, let player terminates at , while and proceed. The corresponding situation can be improved by to , which in its turn can be improved by to . Repeating the similar procedure two times more we obtain the improvement cycle shown in Figure 2.

There are two more situations, which result in and . They appear when all three players terminate or proceed simultaneously. Yet, none of these two situations is an NE either. Moreover, each of them can be improved by every player .

Thus, the following negative result holds, which we recall without proof from [4]; see also [1].

Theorem 7. Game has no uniform NE in pure strategies whenever .

We note that each player has positive payoffs. This is without loss of generality as we can shift the payoffs by a positive constant without changing the game.

A similar two-person uniform NE-free example was suggested in [1], for . Let us consider a family of the utility functions defined by the following constraints:

We claim that the Chess-like game has no uniform NE whenever .

Let us remark that and that is the worst outcome for both players for all . To verify this, let us consider the normal form in Figure 3. By Theorem 5, there is a uniformly BR of player to each strategy of player and vice versa. It is not difficult to check that the obtained two sets of the BRs (which are denoted by the white discs and black squares in Figure 3) are disjoint. Hence, there is no uniform NE. Furthermore, it is not difficult to verify that the obtained situations induce an improvement cycle of length and two improvement paths of lengths and that end in this cycle.

Theorem 8 (see [1]). Game has no uniform NE in pure strategies whenever .

The goal of the present paper is to demonstrate that the above two game structures may have no uniform NE not only in pure but also in mixed strategies. Let us note that by Nash’s theorem [21, 22] NE in mixed strategies exist in any initialized game structure. Yet, this result cannot be extended to the noninitialized game structure and uniform NE. In this research we are motivated by the results of [8, 11].

1.10. Mixed and Independently Mixed Strategies

Standardly, a mixed strategy of a player is defined as a probabilistic distribution over the set of his pure strategies. Furthermore, is called an independently mixed strategy if randomizes in his positions independently. We will denote by and by the sets of mixed and independently mixed strategies of player , respectively.

Remark 9. Let us recall that the players are restricted to their positional strategies and let us also note that the latter concept is closely related to the so-called behavioral strategies introduced by Kuhn [19, 20]. Although Kuhn restricted himself to trees, yet his construction can be extended to directed graphs, too.

Let us recall that a game structure is called play-once if each player is in control of a unique position. For example, is play-once. Obviously, the classes of mixed and independently mixed strategies coincide for a play-once game structure. However, for these two notion differ. Each player controls positions and has pure strategies. Hence, the set of mixed strategies is of dimension , while the set of the independently mixed strategies is only -dimensional.

2. Markovian and A Priori Realizations

For the independently mixed strategies we will consider two different options.

For every player let us consider a probability distribution for all positions , which assigns a probability to each move from , standardly assuming

Now, the limit distributions of the terminals can be defined in two ways, which we will be referred to as the Markovian and a priori realizations.

The first approach is classical; the limit distribution can be found by solving a system of linear equations; see, for example, [27] and also [26].

For example, let us consider and let be the probability to proceed in for . If , then, obviously, the play will cycle with probability resulting in the limit distribution for . Otherwise, assuming that is the initial position, we obtain the limit distribution:

Indeed, positions are transient and the probability of cycling forever is whenever . Obviously, the sum of the above four probabilities is .

The Markovian approach assumes that for the move is chosen randomly, in accordance with the distribution , and independently for all (furthermore, is a fixed initial position). In particular, if the play comes to the same position again, that is, for some , then the moves and may be distinct although they are chosen (independently) with the same distribution .

The concept of a priori realization is based on the following alternative assumptions. A move is chosen according to , independently for all , but only once, before the game starts. Being chosen the move is applied whenever the play comes at . By these assumptions, each infinite play is a lasso; that is, it consists of an initial part (that might be empty) and an infinitely repeated dicycle . Alternatively, may be finite; that is, it terminates in a . In both cases, begins in and the probability of is the product of the probabilities of all its moves, . In this way, we obtain a probability distribution on the set of lassos of the digraph. In particular, the effective payoff is defined as the expected payoffs for the corresponding lassos. Let us also note that (in contrast to the Markovian case) the computation of limit distribution is not computationally efficient, since the set of plays may grow exponentially in size of the digraph. No polynomial algorithm computing the limit distribution is known for a priori realizations. Returning to our example , we obtain the following limit distribution: with initial position . The probability of outcome is ; it is strictly positive whenever for all . Indeed, in contrast to the Markovian realization, the cycle will be repeated infinitely whenever it appears once under a priori realization.

Remark 10. Thus, solving the Chess-like games in the independently mixed strategies looks more natural under a priori (rather than Markovian) realizations. Unfortunately, it seems not that easy to suggest more applications of a priori realizations and we have to acknowledge that the concept of the Markovian realization is much more fruitful. Let us also note that playing in pure strategies can be viewed as a special case of both Markovian and a priori realizations with degenerate probability distributions.

As we already mentioned, the mixed and independently mixed strategies coincide for since it is play-once. Yet, these two classes of strategies differ in .

3. Chess-Like Games with No Uniform NE

In the present paper, we will strengthen Theorems 7 and 8, showing that games and may fail to have an NE (not only in pure, but even) in mixed strategies, as well as in the independently mixed strategies, under both Markovian and a priori realizations.

For convenience, let denote the set of indices of nonterminal positions. We will refer to positions giving only these indices.

Let us recall the definition of payoff function of player for the initial position and the strategy profile ; see Theorem 5. Let us extend this definition introducing the payoff function for the mixed and independently mixed strategies. In both cases, we define it as the expected payoff, under one of the above realizations, and denote by , where is an -vector whose th coordinate is the probability of proceeding (not terminating) at position .

Remark 11. Let us observe that, in both and , the payoff functions , are continuously differentiable functions of when for all , for all players . Hence, if is a uniform NE such that for all (under either a priori or Markovian realization), then
In the next two sections, we will construct games that have no uniform NE under both, a priori and Markovian, realizations. Assuming that a uniform mixed NE exists, we will obtain a contradiction with (7) whenever for all .

3.1. Examples

The next lemma will be instrumental in the proofs of the following two theorems.

Lemma 12. The probabilities to proceed satisfy for all in any independently mixed uniform NE in game , where , and under both a priori and Markovian realizations.

Proof. Let us assume indirectly that there is an (independently) mixed uniform NE under a priori realization with for some . This would imply the existence of an acyclic game with uniform NE, in contradiction with Theorem 7. Now let us consider the case . Due to the circular symmetry of , we can choose any player, say, . The preference list of player is . His most favorable outcome, , is not achievable since . Hence, because his second best outcome is . Thus, the game is reduced to an acyclic one, in contradiction with Theorem 7, again.

Theorem 13. Game has no uniform NE in independently mixed strategies under a priori realization whenever .

Proof. To simplify our notation we denote by and the following and preceding positions along the 3-cycle of , respectively. Assume indirectly that forms a uniform NE and considers the effective payoff of player 1: where is the initial position.
By Lemma 12, we must have for . Therefore (7) must hold. Hence, and follows since . Thus, , in contradiction to our assumption.

Let us recall that for , independently mixed strategies and mixed strategies are the same.

Now, let us consider the Markovian realization. Game may have no NE in mixed strategies under Markovian realization either, yet, only for some special payoffs .

Theorem 14. Game , with , has no uniform NE in independently mixed strategies under Markovian realization if and only if , where

It is easy to verify that for whenever . Let us also note that in the symmetric case the above condition turns into .

Proof. Let be a uniform NE in the game under Markovian realization. Then, by Lemma 12, for . The payoff function of a player, with respect to the initial position that this player controls, is given by one of the next three formulas:
By Lemma 12, (7) holds for any uniform NE. Therefore we have
Setting for , we can transform the above equations to the following form: Assuming , and using successive elimination, we uniquely express via as follows:
Interestingly, all three inequalities are equivalent with the condition , that is, , which completes the proof.