Weak KAM Solutions of a Discrete-Time Hamilton-Jacobi Equation in a Minimax Framework

Abstract

The purpose of this paper is to study the existence of solutions of a Hamilton-Jacobi equation in a minimax discrete-time case and to show different characterizations for a real number called the critical value, which plays a central role in this work. We study the behavior of solutions of this problem using tools of game theory to obtain a “fixed point” of the Lax operator associated, considering some facts of weak KAM theory to interpret these solutions as discrete viscosity solutions. These solutions represent the optimal payoff of a zero-sum game of two players, with increasingly long time payoffs. The developed techniques allow us to study the behavior of an infinite time game without using discount factors or average actions.

1. Introduction

The Hamilton-Jacobi equation is a very important tool for the study of Lagrangian systems and control theory. In the case of convex Lagrangians, Fathi, Mather, and Mañe have perfected several techniques to understand the solutions of this equation ([1–7]). These techniques can be translated to a discrete case where, instead of a Lagrangian, there is a function 𝑉(𝑥,𝑦) of two variables in the configuration space.

The aim of this paper is to study a discrete-time system that emulates a convex and concave Lagrangian. We need to study the behavior of solutions of a discrete-time version of a Hamilton-Jacobi equation in the context of game theory. A real number, called the critical value, plays a central role in this work; this number is the asymptotic average action of optimal trajectories. We study the existence and characterization of solutions of a Hamilton-Jacobi equation for this kind of game. A relevant aspect is that these techniques allow us to study infinite-time games without using discount factors or average actions.

Consider the compact metric spaces 𝑀 and 𝑁, which are the state spaces for players 𝐽_𝑀 and 𝐽_𝑁, respectively. For (𝑥,𝑦)∈𝑀×𝑁, a continuous function 𝑔∶𝑀×𝑁→ℝ, and a Lipschitz function 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ, we define the static game payoff by 𝑃_𝑔𝑥₁,𝑦₁∶=𝑔𝑥₁,𝑦₁−𝑉𝑥,𝑦,𝑥₁,𝑦₁. (1) The function 𝑃 is the amount that player 𝐽_𝑀 obtains from 𝐽_𝑁 and vice versa; so player 𝐽_𝑀 wants to maximize the payoff 𝑃, while player 𝐽_𝑁 wants to minimize it. The lower game value is defined by ℒ¹𝑔𝑥,𝑦∶=sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑃_𝑔𝑥₁,𝑦₁. (2) It is observed in [8] that it is possible to express the lower game value as follows: ℒ¹𝑔𝑥,𝑦=inf_𝜃(𝑧)sup_𝑧∈𝑀𝑃_𝑔(𝑧,𝜃(𝑧)), (3) where the function 𝜃∶𝑀→𝑁 is called a strategy for player 𝐽_𝑁. These functions are very important in this work and will be defined in general for a zero-sum game with finite horizon.

For 𝑛∈ℤ⁺, consider the finite horizon problem, where players 𝐽_𝑀 and 𝐽_𝑁 alternate their moves beginning at initial state (𝑥₀,𝑦₀) and setting with them the state trajectories determined by finite sequences 𝑥(𝑛)⊂𝑀 and 𝑦(𝑛)⊂𝑁, denoted by 𝑥(𝑛)∶={𝑥₀,𝑥₁,…,𝑥_𝑛} and 𝑦(𝑛)∶={𝑦₀,𝑦₁,…,𝑦_𝑛}, respectively. The state of the system depends on the initial condition (𝑥₀,𝑦₀) and the players decisions to get from (𝑥_𝑖,𝑦_𝑖) to (𝑥_𝑖+1,𝑦_𝑖+1), where 𝑖=0,1,…,𝑛−1. For (𝑥,𝑦)∈𝑀×𝑁, we will denote by 𝑆_𝑀(𝑥,𝑛)∶=𝑥(𝑛)=𝑥_𝑖^𝑛_𝑖=0∶𝑥₀,𝑆=𝑥_𝑁𝑦,𝑛∶=𝑦(𝑛)=𝑦_𝑖^𝑛_𝑖=0∶𝑦₀=𝑦, (4) the sets of finite sequences in 𝑀 and 𝑁, with initial states 𝑥∈𝑀 and 𝑦∈𝑁. We define the action by 𝐴^𝑛𝑥(𝑛),𝑦(𝑛)∶=^𝑛−1_𝑖=0𝑉𝑥_𝑖,𝑦_𝑖,𝑥_𝑖+1,𝑦_𝑖+1, (5) and the game payoff by 𝑃_𝑔𝑥(𝑛),𝑦(𝑛)∶=𝑔𝑥_𝑛,𝑦_𝑛−𝐴^𝑛𝑥(𝑛),𝑦(𝑛). (6)

Then, the lower game value with finite horizon 𝑛 and initial state (𝑥,𝑦)∈𝑀×𝑁 is ℒ^𝑛𝑔𝑥,𝑦∶=sup_{𝑥𝑛∈𝑀}inf_{𝑦𝑛∈𝑁}sup_{𝑥𝑛−1∈𝑀}inf_{𝑦𝑛−1∈𝑁}⋯sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑃_𝑔𝑥(𝑛),𝑦(𝑛). (7) However, ℒ^𝑛 can be defined directly using progressive strategies (see [8–10]). The idea is to use a tool that allows player 𝐽_𝑁 to select one point 𝑦_𝑖 knowing the current and past opponent choices, without knowing the future choices. To this end, we define a progressive strategy Θ(𝑛) for player 𝐽_𝑁 as a function Θ(𝑛)∶𝑆_𝑀(𝑥,𝑛)→𝑆_𝑁(𝑦,𝑛) which satisfies the following property: for each 𝑚∈ℤ such that 0≤𝑚≤𝑛 and 𝑥(𝑛),𝑧(𝑛)∈𝑆_𝑀(𝑥,𝑛), we have 𝑥_𝑖=𝑧_𝑖(𝑛,0≤𝑖≤𝑚⟹Θ)[𝑥(𝑛)]_𝑖=Θ(𝑛)[𝑧(𝑛)]_𝑖,0≤𝑖≤𝑚, (8) where Θ(𝑛)[𝑥(𝑛)]_𝑖 is the 𝑖th element of sequence Θ(𝑛)[𝑥(𝑛)]∈𝑆_𝑁(𝑦,𝑛). The set of progressive strategies for player 𝐽_𝑁 is denoted by ℰ_𝑁Θ(𝑛)𝑥,𝑦,𝑛∶=∶𝑆_𝑀(𝑥,𝑛)⟶𝑆_𝑁𝑦,𝑛progressivestrategy. (9) Finally, the lower game value can be expressed as ℒ^𝑛𝑔𝑥,𝑦=inf_Θ(𝑛)sup_𝑥(𝑛)𝑃_𝑔(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]), (10) where 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛) and Θ(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛). This function is called Lax operator and will be essential for understanding the asymptotic behavior of game payoffs. The Proposition 6 shows that ℒ^𝑛𝑔𝑥,𝑦=ℒ¹∘⋯∘ℒ¹_𝑛times𝑔𝑥,𝑦. (11)

In the next section of this paper, the objective is to prove the existence of a kind of “fixed point” of the Lax operator. We will adapt the methods developed in the work of Fathi ([3]) about weak KAM theory, now in a minimax context of game theory. It will be necessary to prove that the Lax operator ℒ^𝑛, defined as in (10), satisfies some important properties: the semigroup property that is also call Dynamic Programming Principle (see [8–10]), a regularity property, monotony, and weak contraction.

Theorem 1. If 𝑀 and 𝑁 are compact metric spaces and 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, then there exist a Lipschitz function 𝑢∈𝐶(𝑀×𝑁,ℝ) and a unique constant 𝑐∈ℝ, such that ℒ^𝑛𝑢=𝑢+𝑛𝑐. (12)

The functions in the previous result are called fixed points of the Lax operator with critical value 𝑐. These fixed points are solutions of the following discrete-time Hamilton-Jacobi equation associated to 𝑉: sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑢𝑥₁,𝑦₁−𝑢𝑥,𝑦−𝑉𝑥,𝑦,𝑥₁,𝑦₁=𝑐. (13) See [11] for a minimal discrete-time case.

In the third section, we will consider an infinite horizon problem, which will help us to find a different characterization of critical value 𝑐. To this end, we define the lower Peierls barrier ℎ⁻_𝑘 as follows, given a real number 𝑘 and an initial state (𝑥,𝑦)∈𝑀×𝑁: ℎ⁻_𝑘𝑥,𝑦∶=liminf_𝑛→∞sup_Θ(𝑛)inf_𝑥(𝑛)𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)])+𝑛𝑘, (14) where 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛) and Θ(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛), see [3, 12, 13].

It is possible to prove that there exists a number 𝑐⁻∈ℝ, such that 𝑐⁻=inf𝑘∈ℝ∶ℎ⁻_𝑘=+∞=sup𝑘∈ℝ∶ℎ⁻_𝑘=−∞, (15) where the value of ℎ⁻_𝑘 does not depend on (𝑥,𝑦)∈𝑀×𝑁. The critical value 𝑐 has a new interpretation characterized by the following result.

Theorem 2. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and (𝑥,𝑦)∈𝑀×𝑁, then ℎ⁻_𝑐−(𝑥,𝑦)∈ℝ and 𝑐⁻=𝑐, where 𝑐 is the critical value.

Analogously, we can define the upper Peierls barrier ℎ⁺_𝑘 by ℎ⁺_𝑘𝑥,𝑦∶=limsup_𝑛→∞sup_Θ(𝑛)inf_𝑥(𝑛)𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)])+𝑛𝑘, (16) for which we can obtain similar results.

Considering that ℎ⁻_𝑐(𝑥,𝑦)∈ℝ and ℎ⁺_𝑐(𝑥,𝑦)∈ℝ, for every (𝑥,𝑦)∈𝑀×𝑁, we prove that sup_𝑛∈ℤ+𝑛𝑐+sup_Θ(𝑛)inf_𝑥(𝑛)𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)])<+∞, (17) and therefore, −𝑐=lim_𝑛→∞sup_Θ(𝑛)inf_𝑥(𝑛)1𝑛𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]). (18)

In addition, using a solution of the Hamilton-Jacobi equation, we can find an optimal strategy, from which optimal sequences are obtained for both players for all times. In particular, we prove the following result.

Theorem 3. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and (𝑥,𝑦)∈𝑀×𝑁, then there are sequences {𝑥^∗_𝑖}^∞_𝑖=0⊂𝑀 and {𝑦^∗_𝑖}^∞_𝑖=0⊂𝑁 such that sup_𝑛∈ℤ+𝑛𝑐+𝐴^𝑛𝑥^∗(𝑛),𝑦^∗(𝑛)<+∞, (19) where 𝑥^∗(𝑛)∶={𝑥^∗_𝑖}^𝑛_𝑖=0∈𝑆_𝑀(𝑥,𝑛) and 𝑦^∗(𝑛)∶={𝑦^∗_𝑖}^𝑛_𝑖=0∈𝑆_𝑁(𝑦,𝑛), where 𝑐 is the critical value.

From this, it follows immediately that −𝑐=lim_𝑛→∞1𝑛𝐴^𝑛𝑥^∗(𝑛),𝑦^∗(𝑛). (20)

The sequences obtained in the previous result satisfy the Bellman principle of optimality, because from each state (𝑥^∗_𝑖,𝑦^∗_𝑖) in the optimal trajectory, the complement remains optimal. The optimality of the remaining decisions from (𝑥^∗_𝑖,𝑦^∗_𝑖) depends on this last state, which is the result of the previous decisions.

In last section, we will show another characterization for the solutions of the Hamilton-Jacobi equation.

For 𝑘∈ℝ, a function 𝑔∈𝐶(𝑀×𝑁,ℝ) is called 𝑘-supersolution if ℒ¹𝑔≥𝑔+𝑘. (21) Analogously, a function 𝑔∈𝐶(𝑀×𝑁,ℝ) is called 𝑘-subsolution if ℒ¹𝑔≤𝑔+𝑘. (22) The function 𝑔 is a 𝑘-solution if 𝑔 is 𝑘-supersolution and 𝑘-subsolution (see [4]).

Clearly, 𝑔 is a 𝑘-solution if and only if 𝑔 is a fixed point of the Lax operator with critical value 𝑘. According to Theorem 1, if there exists a unique constant for fixed points of the Lax operator, then there is only one kind of 𝑘-solutions, in fact for 𝑘=𝑐. The 𝑐-solutions are called critical solutions. From this, we get another characterization of 𝑐, which appears in the following result.

Theorem 4. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and 𝑢∈𝐶(𝑀×𝑁,ℝ) is a fixed point of the Lax operator ℒ^𝑛 with critical value 𝑐∈ℝ, then 𝑐=sup𝑆_𝑈(𝑢)=inf𝑆_𝐿(𝑢), (23) where 𝑆_𝑈(𝑢),𝑆∶=𝑘∈ℝ∶𝑢is𝑘-𝑠𝑢𝑝𝑒𝑟𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛_𝐿(𝑢)∶={𝑘∈ℝ∶𝑢is𝑘-𝑠𝑢𝑏𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛}. (24)

Finally, we will show that −ℎ⁻_𝑐 is a critical solution and therefore a solution of the Hamilton-Jacobi equation.

2. Lax Operator

Let 𝑀 and 𝑁 be compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ a Lipschitz function, and 𝑛∈ℤ⁺. Consider a zero-sum game of two players, 𝐽_𝑀 and 𝐽_𝑁, with finite horizon 𝑛, where 𝑀 and 𝑁 represent the state spaces and sequences 𝑥(𝑛)={𝑥₀,𝑥₁,…,𝑥_𝑛}⊂𝑀 and 𝑦(𝑛)={𝑦₀,𝑦₁,…,𝑦_𝑛}⊂𝑁 determine the state trajectories, respectively, for players 𝐽_𝑀 and 𝐽_𝑁. The function 𝐴^𝑛(𝑥(𝑛),𝑦(𝑛)), defined by 𝐴^𝑛𝑥(𝑛),𝑦(𝑛)∶=^𝑛−1_𝑖=0𝑉𝑥_𝑖,𝑦_𝑖,𝑥_𝑖+1,𝑦_𝑖+1, (25) is the game action. For 𝑔∈𝐶(𝑀×𝑁,ℝ) and (𝑥,𝑦)∈𝑀×𝑁, the game payoff 𝑃_𝑔∶𝑆_𝑀(𝑥,𝑛)×𝑆_𝑁(𝑦,𝑛)→ℝ is defined by 𝑃_𝑔𝑥(𝑛),𝑦(𝑛)∶=𝑔𝑥_𝑛,𝑦_𝑛−𝐴^𝑛𝑥(𝑛),𝑦(𝑛). (26) If ℰ_𝑁(𝑥,𝑦,𝑛) is the set of progressive strategies for player 𝐽_𝑁, the Lax operator is the lower game value, defined by ℒ^𝑛𝑔𝑥,𝑦∶=inf_Θ(𝑛)sup_𝑥(𝑛)𝑃_𝑔(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]), (27) with 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛) and Θ(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛). We prove the following regularity property for the Lax operator.

Lemma 5. The Lax operator ℒ^𝑛𝑔 is 𝐾-Lipschitz, for any 𝑛∈ℤ⁺ and 𝑔∈𝐶(𝑀×𝑁,ℝ), where 𝐾 is the Lipschitz constant of 𝑉.

Proof. We will consider (𝑥,𝑦),(𝑧,𝑤)∈𝑀×𝑁, 𝑔∈𝐶(𝑀×𝑁,ℝ), and 𝜀>0. There exists Θ¹(𝑛)∈ℰ_𝑁(𝑧,𝑤,𝑛) such that ℒ^𝑛𝑔(𝑧,𝑤)>sup_𝑧(𝑛)𝑃_𝑔𝑧(𝑛),Θ¹(𝑛)[𝑧(𝑛)]−𝜀, (28) where 𝑧(𝑛)∈𝑆_𝑀(𝑧,𝑛).
For a finite sequence 𝑥(𝑛)={𝑥,𝑥₁,…,𝑥_𝑛}∈𝑆_𝑀(𝑥,𝑛), we define Θ²(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛) as Θ²(𝑛)[𝑥(𝑛)]_𝑖=Θ𝑦,𝑖=0₁(𝑛)𝑧,𝑥₁,…,𝑥_𝑛𝑖,1≤𝑖≤𝑛. (29)
For this strategy, we have ℒ^𝑛𝑔𝑥,𝑦≤sup_𝑥(𝑛)𝑃_𝑔𝑥(𝑛),Θ²(𝑛)[𝑥(𝑛)], (30) where 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛). It follows that there is 𝑥^∗(𝑛)={𝑥,𝑥^∗₁,…,𝑥^∗_𝑛} such that ℒ^𝑛𝑔𝑥,𝑦<𝑃_𝑔𝑥^∗(𝑛),Θ²(𝑛)𝑥^∗(𝑛)+𝜀. (31)
We define 𝑧^∗(𝑛)∈𝑆_𝑀(𝑧,𝑛) as 𝑧^∗(𝑛)={𝑧,𝑥^∗₁,…,𝑥^∗_𝑛}. Combining (28) with (31), we obtain ℒ^𝑛𝑔𝑥,𝑦−ℒ^𝑛𝑔(𝑧,𝑤)<𝑃_𝑔𝑥^∗(𝑛),Θ²(𝑛)𝑥^∗(𝑛)−𝑃_𝑔𝑧^∗(𝑛),Θ¹(𝑛)𝑧^∗(𝑛)+2𝜀. (32) Since 𝑥^∗_𝑖=𝑧^∗_𝑖 and Θ²(𝑛)[𝑥^∗(𝑛)]_𝑖=Θ¹(𝑛)[𝑧^∗(𝑛)]_𝑖, for 1≤𝑖≤𝑛, we have ℒ^𝑛𝑔𝑥,𝑦−ℒ^𝑛𝑔(𝑧,𝑤)<𝑉𝑧,𝑤,𝑥^∗₁,Θ¹(𝑛)𝑧^∗(𝑛)₁−𝑉𝑥,𝑦,𝑥^∗₁,Θ¹(𝑛)𝑧^∗(𝑛)₁,(+2𝜀≤𝐾𝑑𝑥,𝑦𝑧,𝑤)+2𝜀, (33) where 𝐾 is the Lipschitz constant of 𝑉 and 𝑑 is a distance in 𝑀×𝑁. Similarly, we obtain the opposite inequality. Thus, ℒ^𝑛𝑔𝑥,𝑦−ℒ^𝑛𝑔(𝑧,𝑤)≤𝐾𝑑𝑥,𝑦,(𝑧,𝑤). (34)

From above result, we prove the semigroup property for the Lax operator, also called the Principle of Dynamic Programming.

Proposition 6 (Semigroup Property). For 𝑚,𝑛∈ℤ⁺, ℒ^𝑚+𝑛𝑔𝑥,𝑦=ℒ^𝑚∘ℒ^𝑛𝑔𝑥,𝑦=inf_Θ(𝑚)sup_𝑥(𝑚)ℒ^𝑛𝑔𝑥_𝑚,Θ(𝑚)[𝑥(𝑚)]_𝑚−𝐴^𝑚(𝑥(𝑚),Θ(𝑚)[𝑥(𝑚)]), (35) where Θ(𝑚)∈ℰ_𝑁(𝑥,𝑦,𝑚) and 𝑥(𝑚)∈𝑆_𝑀(𝑥,𝑚).

Proof. We will consider 𝜀>0 and 𝑔∈𝐶(𝑀×𝑁,ℝ). To prove the relation ℒ^𝑚+𝑛𝑔=ℒ^𝑚∘ℒ^𝑛𝑔, we will see that, for every (𝑥,𝑦)∈𝑀×𝑁, the inequalities ℒ^𝑚+𝑛𝑔(𝑥,𝑦)≤ℒ^𝑚∘ℒ^𝑛𝑔(𝑥,𝑦) and ℒ^𝑚+𝑛𝑔(𝑥,𝑦)≥ℒ^𝑚∘ℒ^𝑛𝑔(𝑥,𝑦) take place simultaneously.
(I) Let us first prove that ℒ^𝑚+𝑛𝑔≤ℒ^𝑚∘ℒ^𝑛𝑔. To do this, take Θ¹(𝑚)∈ℰ_𝑁(𝑥,𝑦,𝑚) such that ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦>sup_𝑥(𝑚)ℒ^𝑛𝑔𝑥_𝑚,Θ¹(𝑚)[𝑥(𝑚)]_𝑚−𝐴^𝑚𝑥(𝑚),Θ¹(𝑚)[𝑥(𝑚)]−𝜀, (36) where 𝑥(𝑚)∈𝑆_𝑀(𝑥,𝑚).
On the other hand, for each (𝑧,𝑤)∈𝑀×𝑁, ℒ^𝑛𝑔(𝑧,𝑤)=inf_Θ(𝑛)sup_𝑧(𝑛)𝑔𝑧_𝑛,Θ(𝑛)[𝑧(𝑛)]_𝑛−𝐴^𝑛(𝑧(𝑛),Θ(𝑛)[𝑧(𝑛)]), (37) where Θ(𝑛)∈ℰ_𝑁(𝑧,𝑤,𝑛) and 𝑧(𝑛)∈𝑆_𝑀(𝑧,𝑛). Hence, there is Θ²_(𝑧,𝑤)(𝑛)∈ℰ_𝑁(𝑧,𝑤,𝑛) such that ℒ^𝑛𝑔(𝑧,𝑤)>sup_𝑧(𝑛)𝑔𝑧_𝑛,Θ²_(𝑧,𝑤)(𝑛)[𝑧(𝑛)]_𝑛−𝐴^𝑛𝑧(𝑛),Θ²_(𝑧,𝑤)(𝑛)[𝑧(𝑛)]−𝜀. (38)
We will define a Θ^∗(𝑚+𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑚+𝑛) using the previous strategy Θ¹(𝑚) and the family Θ²_(𝑧,𝑤)(𝑛). Given a sequence in 𝑆_𝑀(𝑥,𝑚+𝑛), the associated sequence in 𝑆_𝑁(𝑦,𝑚+𝑛) is defined as follows: the fist 𝑚 terms are given by the Θ¹(𝑚) strategy and the second part is given by Θ²_{(𝑧∗,𝑤∗)}(𝑛), for a specific (𝑧^∗,𝑤^∗). More precisely, for 𝑥(𝑚+𝑛)∈𝑆_𝑀(𝑥,𝑚+𝑛), Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑖=Θ¹(𝑚)𝑥¹(𝑚)_𝑖Θ,0≤𝑖≤𝑚_{2(𝑧∗,𝑤∗)}(𝑛)𝑥²(𝑛)_𝑖−𝑚,𝑚+1≤𝑖≤𝑚+𝑛, (39) where 𝑥¹(𝑚)={𝑥,𝑥₁,…,𝑥_𝑚}, (𝑧^∗,𝑤^∗)=(𝑥_𝑚,Θ¹(𝑚)[𝑥¹(𝑚)]_𝑚), 𝑥²(𝑛)={𝑧^∗,𝑥_𝑚+1,…,𝑥_𝑚+𝑛}, and Θ²_{(𝑧∗,𝑤∗)}(𝑛)∈ℰ_𝑁(𝑧^∗,𝑤^∗,𝑛). Taking any 𝑥(𝑚+𝑛)∈𝑆_𝑀(𝑥,𝑚+𝑛) and combining (36) with (38), we obtain ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦>ℒ^𝑛𝑔𝑥_𝑚,Θ¹(𝑚)𝑥¹(𝑚)_𝑚−𝐴^𝑚𝑥¹(𝑚),Θ¹(𝑚)𝑥¹(𝑚𝑥)−𝜀>𝑔_𝑚+𝑛,Θ²_{(𝑧∗,𝑤∗)}(𝑛)𝑥²(𝑛)_𝑛−𝐴^𝑛𝑥²(𝑛),Θ²_{(𝑧∗,𝑤∗)}(𝑛)𝑥²(𝑛)−𝐴^𝑚𝑥¹(𝑚),Θ¹(𝑚)𝑥¹(𝑚𝑥)−2𝜀=𝑔_𝑚+𝑛,Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑚+𝑛−𝐴^𝑚+𝑛𝑥(𝑚+𝑛),Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]−2𝜀. (40) Therefore, sup_{𝑥()𝑚+𝑛}𝑔𝑥_𝑚+𝑛,Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑚+𝑛−𝐴^𝑚+𝑛𝑥(𝑚+𝑛),Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]≤ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦+2𝜀, (41) with 𝑥(𝑚+𝑛)∈𝑆_𝑀(𝑥,𝑚+𝑛). This implies that ℒ^𝑚+𝑛𝑔𝑥,𝑦=inf_{Θ()𝑚+𝑛}sup_{𝑥()𝑚+𝑛}𝑔𝑥_𝑚+𝑛,Θ(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑚+𝑛−𝐴^𝑚+𝑛(𝑥(𝑚+𝑛),Θ(𝑚+𝑛)[𝑥(𝑚+𝑛)])≤ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦+2𝜀, (42) and consequently ℒ^𝑚+𝑛𝑔≤ℒ^𝑚∘ℒ^𝑛𝑔.
(II) It remains to prove that ℒ^𝑚+𝑛𝑔≥ℒ^𝑚∘ℒ^𝑛𝑔. To this end, we will consider Θ^∗(𝑚+𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑚+𝑛) such that ℒ^𝑚+𝑛𝑔𝑥,𝑦>sup_{𝑥()𝑚+𝑛}𝑔𝑥_𝑚+𝑛,Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑚+𝑛−𝐴^𝑚+𝑛𝑥(𝑚+𝑛),Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]−𝜀, (43) where 𝑥(𝑚+𝑛)∈𝑆_𝑀(𝑥,𝑚+𝑛).
We will define Θ¹(𝑚)∈ℰ_𝑁(𝑥,𝑦,𝑚) as the 𝑚-time restriction of Θ^∗(𝑚+𝑛). Given a sequence 𝑧(𝑚)∈𝑆_𝑀(𝑥,𝑚), the associated sequence in 𝑆_𝑁(𝑦,𝑚) is defined as follows: the 𝑚 terms are given by the fist 𝑚 terms of Θ^∗(𝑚+𝑛) strategy applied to some extension of 𝑧(𝑚) in 𝑆_𝑀(𝑥,𝑚+𝑛), adding any sequence 𝑤(𝑛)∈𝑆_𝑀(𝑧_𝑚,𝑛). Since Θ^∗(𝑚+𝑛) is a strategy, it follows that the function Θ¹(𝑚) is well defined and it is a strategy. More precisely, for 𝑧(𝑚)∈𝑆_𝑀(𝑥,𝑚), taking any 𝑤(𝑛)={𝑧_𝑚,𝑤₁,…,𝑤_𝑛}∈𝑆_𝑀(𝑧_𝑚,𝑛), we define Θ¹(𝑚)∈ℰ_𝑁(𝑥,𝑦,𝑚) as Θ¹(𝑚)[𝑧(𝑚)]_𝑖=Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑖, (44) for 𝑖=0,1,…,𝑚, where 𝑥(𝑚+𝑛)={𝑥,𝑧₁,…,𝑧_𝑚,𝑤₁,…,𝑤_𝑛}. So ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦≤sup_𝑧(𝑚)ℒ^𝑛𝑔𝑧_𝑚,Θ¹(𝑚)[𝑧(𝑚)]_𝑚−𝐴^𝑚𝑧(𝑚),Θ¹(𝑚)[𝑧(𝑚)], (45) where 𝑧(𝑚)∈𝑆_𝑀(𝑥,𝑚). Consequently, there is 𝑧¹(𝑚)∈𝑆_𝑀(𝑥,𝑚) such that ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦<ℒ^𝑛𝑔𝑧¹_𝑚,Θ¹(𝑚)𝑧¹(𝑚)_𝑚−𝐴^𝑚𝑧¹(𝑚),Θ¹(𝑚)𝑧¹(𝑚)+𝜀. (46) Once that 𝑧¹_𝑚∈𝑀 is fixed so will be Θ¹(𝑚)[𝑧¹(𝑚)]_𝑚∈𝑁. We now define Θ²(𝑛)∈ℰ_𝑁(𝑧¹_𝑚,Θ¹(𝑚)[𝑧¹(𝑚)]_𝑚,𝑛), for 𝑤(𝑛)∈𝑆_𝑀(𝑧¹_𝑚,𝑛), as Θ²(𝑛)[𝑤(𝑛)]_𝑖=Θ^∗(𝑚+𝑛)[𝑥(𝑚+𝑛)]_𝑖+𝑚, (47) for 𝑖=0,1,…,𝑛, where 𝑥(𝑚+𝑛)={𝑥,𝑧¹₁,…,𝑧¹_𝑚,𝑤₁,…,𝑤_𝑛}. We have ℒ^𝑛𝑔𝑧¹_𝑚,Θ¹(𝑚)𝑧¹(𝑚)_𝑚≤sup_𝑤(𝑛)𝑔𝑤_𝑛,Θ²(𝑛)[𝑤(𝑛)]_𝑛−𝐴^𝑛𝑤(𝑛),Θ²(𝑛)[𝑤(𝑛)], (48) where 𝑤(𝑛)∈𝑆_𝑀(𝑧¹_𝑚,𝑛). Therefore, there is 𝑧²(𝑛)∈𝑆_𝑀(𝑧¹_𝑚,𝑛) such that ℒ^𝑛𝑔𝑧¹_𝑚,Θ¹(𝑚)𝑧¹(𝑚)_𝑚<𝑔𝑧²_𝑛,Θ²(𝑛)𝑧²(𝑛)_𝑛−𝐴^𝑛𝑧²(𝑛),Θ²(𝑛)𝑧²(𝑛)+𝜀. (49) Define 𝑥^∗(𝑚+𝑛)∈𝑆_𝑀(𝑥,𝑚+𝑛) as 𝑥^∗(𝑚+𝑛)=𝑥,𝑧¹₁,…,𝑧¹_𝑚,𝑧²₁,…,𝑧²_𝑛. (50) Combining (43) and (46) with (49) we obtain: ℒ^𝑚ℒ^𝑛𝑔𝑥,𝑦<ℒ^𝑛𝑔𝑧¹_𝑚,Θ¹(𝑚)𝑧¹(𝑚)_𝑚−𝐴^𝑚𝑧¹(𝑚),Θ¹(𝑚)𝑧¹(𝑚𝑧)+𝜀<𝑔²_𝑛,Θ²(𝑛)𝑧²(𝑛)_𝑛−𝐴^𝑛𝑧²(𝑛),Θ²(𝑛)𝑧²(𝑛)−𝐴^𝑚𝑧¹(𝑚),Θ¹(𝑚)𝑧¹(𝑚𝑥)+2𝜀=𝑔^∗_𝑚+𝑛,Θ^∗(𝑚+𝑛)𝑥^∗(𝑚+𝑛)_𝑚+𝑛−𝐴^𝑚+𝑛𝑥^∗(𝑚+𝑛),Θ^∗(𝑚+𝑛)𝑥^∗(𝑚+𝑛)+2𝜀<ℒ^𝑚+𝑛𝑔𝑥,𝑦+3𝜀, (51) for this reason ℒ^𝑚∘ℒ^𝑛≤ℒ^𝑚+𝑛.

Lemma 7. If 𝑔,𝑓∈𝐶(𝑀×𝑁,ℝ), then(1)(monotony) if 𝑔≤𝑓, then ℒ^𝑛𝑔≤ℒ^𝑛𝑓;(2)if 𝑘∈ℝ, then ℒ^𝑛(𝑘+𝑔)=𝑘+ℒ^𝑛𝑔;(3)(weak contraction) ‖ℒ^𝑛𝑔−ℒ^𝑛𝑓‖_∞≤‖𝑔−𝑓‖_∞.

Proof. The first two properties are a direct consequence of the definition. For last the claim, choose (𝑥,𝑦)∈𝑀×𝑁, then 𝑓𝑥,𝑦+𝑔−𝑓_∞≤𝑔𝑥,𝑦≤𝑓𝑥,𝑦+𝑔−𝑓_∞. (52) The previous items imply that ℒ^𝑛𝑓𝑥,𝑦+𝑔−𝑓_∞≤ℒ^𝑛𝑔𝑥,𝑦≤ℒ^𝑛𝑓𝑥,𝑦+𝑔−𝑓_∞. (53) So |(ℒ^𝑛𝑔−ℒ^𝑛𝑓)(𝑥,𝑦)|≤‖𝑔−𝑓‖_∞, for every (𝑥,𝑦)∈𝑀×𝑁. As a result, ℒ^𝑛𝑔−ℒ^𝑛𝑓_∞≤𝑔−𝑓_∞. (54)

Theorem 1 is analogue to Weak KAM Theorem. We use the same argument of Fathi (see [3]) to prove the existence of fixed points of this discrete-time case of the Lax operator.

According to this result, we say that 𝑢 is a fixed point of the Lax operator with critical value 𝑐.

Proof. (I) To prove the existence of a fixed point, we will use the same argument used by Fathi ([3]) for the continuous-time case.
Let 𝐸 be the quotient of the set of continuous functions 𝐶(𝑀×𝑁,ℝ) modulo constants, 𝐸=𝐶(𝑀×𝑁,ℝ)/ℝ⋅1. The norm ‖⋅‖∶𝐸→ℝ is defined as ‖[𝑢]‖=inf_𝑎∈ℝ‖𝑢+𝑎‖_∞, (55) for [𝑢]∈𝐸. Then, (𝐸,‖⋅‖) is a Banach space. We define the operator ℒ^𝑛∶𝐸→𝐸 as ℒ^𝑛[𝑢]=ℒ^𝑛𝑢, (56) which satisfies the weak contraction property. Thus, [𝑢]↦𝜆ℒ^𝑛[𝑢], with 𝜆∈(0,1), is a contraction; so there are fixed points [𝑢]_𝜆∈𝐸 of 𝜆ℒ^𝑛. As ℒ^𝑛 is equi-Lipschitz, since Arzela-Ascoli theorem, it follows that {[𝑢]_𝜆} has a uniformly convergent subsequence when 𝜆→1. Then, ℒ^𝑛 has a fixed point [𝑢]_∗∈𝐸, which in terms of ℒ^𝑛 means that there is a constant 𝑐_𝑛∈ℝ such that ℒ^𝑛𝑢_∗=𝑢_∗+𝑐_𝑛, (57) for each positive integer 𝑛. From the semigroup property, there is 𝑐∈ℝ such that ℒ^𝑛𝑢_∗=𝑢_∗+𝑛𝑐, (58) for all positive integers 𝑛.
Function 𝑢_∗ is a Lipschitz function, because ℒ^𝑛𝑢_∗ is also a Lipschitz function. In fact, according to Lemma 5, 𝑢_∗ is 𝐾-Lipschitz, where 𝐾 is the Lipschitz constant of 𝑉.
(II) It remains to prove that the number 𝑐 is unique. Let 𝜀 be positive. Suppose there are 𝑢_𝑖∈𝐶(𝑀×𝑁,ℝ) and 𝑐_𝑖∈ℝ, 𝑖=1,2, such that ℒ^𝑛𝑢_𝑖=𝑢_𝑖+𝑛𝑐_𝑖, with 𝑐₁>𝑐₂. Choose Θ^∗(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛) such that ℒ^𝑛𝑢₂𝑥,𝑦>sup_𝑥(𝑛)𝑢₂𝑥_𝑛,Θ^∗(𝑛)[𝑥(𝑛)]_𝑛−𝐴^𝑛𝑥(𝑛),Θ^∗(𝑛)[𝑥(𝑛)]−𝜀. (59) For this strategy, we also have that ℒ^𝑛𝑢₁𝑥,𝑦≤sup_𝑥(𝑛)𝑢₁𝑥_𝑛,Θ^∗(𝑛)[𝑥(𝑛)]_𝑛−𝐴^𝑛𝑥(𝑛),Θ^∗(𝑛)[𝑥(𝑛)]. (60) So there is 𝑥^∗(𝑛)∈𝑆_𝑀(𝑥,𝑛), such that ℒ^𝑛𝑢₁𝑥,𝑦<𝑢₁𝑥^∗_𝑛,Θ^∗(𝑛)𝑥^∗(𝑛)_𝑛−𝐴^𝑛𝑥^∗(𝑛),Θ^∗(𝑛)𝑥^∗(𝑛)+𝜀. (61) Therefore, using the hypothesis, we obtain 𝑢₁𝑥,𝑦+𝑛𝑐₁<𝑢₁𝑥^∗_𝑛,Θ^∗(𝑛)𝑥^∗(𝑛)_𝑛−𝐴^𝑛𝑥^∗(𝑛),Θ^∗(𝑛)𝑥^∗(𝑛𝑢)+𝜀,₂𝑥,𝑦+𝑛𝑐₂>𝑢₂𝑥^∗_𝑛,Θ^∗(𝑛)𝑥^∗(𝑛)_𝑛−𝐴^𝑛𝑥^∗(𝑛)Θ^∗(𝑛)𝑥^∗(𝑛)−𝜀. (62) Considering the subtraction of the previous inequalities, we obtain following relations: inf_𝑀×𝑁𝑢₁−𝑢₂+𝑛𝑐₁−𝑐₂≤𝑢₁𝑥,𝑦+𝑛𝑐₁−𝑢₂𝑥,𝑦−𝑛𝑐₂<𝑢₁𝑥^∗_𝑛,Θ^∗(𝑚+𝑛)𝑥^∗(𝑛)_𝑛−𝑢₂𝑥^∗_𝑛,Θ^∗(𝑚+𝑛)𝑥^∗(𝑛)_𝑛+2𝜀≤sup_𝑀×𝑁𝑢₁−𝑢₂+2𝜀, (63) then 𝑛𝑐₁−𝑐₂<sup_𝑀×𝑁𝑢₁−𝑢₂−inf_𝑀×𝑁𝑢₁−𝑢₂+2𝜀. (64) On the other hand, because 𝑐₁>𝑐₂, we have 𝑛(𝑐₁−𝑐₂)→∞ when 𝑛→∞, which is a contradiction.

Considering the one-step case (see [8]), we can rewrite the Lax operator as ℒ¹𝑔𝑥,𝑦=sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑔𝑥₁,𝑦₁−𝑉𝑥,𝑦,𝑥₁,𝑦₁. (65) If 𝑢∈𝐶(𝑀×𝑁,ℝ) is a fixed point with critical value 𝑐, then 𝑢 is a solution for the following discrete-time analog of the Hamilton-Jacobi equation: sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑢𝑥₁,𝑦₁−𝑢𝑥,𝑦−𝑉𝑥,𝑦,𝑥₁,𝑦₁=𝑐. (66)

Conversely, if 𝑢 is a solution of the previous Hamilton-Jacobi equation, then 𝑢 is a fixed point of the Lax operator ℒ¹ with critical value 𝑐. According to semigroup property and relation 2 in Lemma 7, it follows that 𝑢 is a fixed point of ℒ^𝑛 with critical value 𝑐, for all 𝑛∈ℤ⁺.

3. Peierls Barrier

Let us now study an infinite horizon problem for players 𝐽_𝑀 and 𝐽_𝑁. Let 𝑀 and 𝑁 be compact metric spaces, and let 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ be a Lipschitz function. For 𝑛∈ℤ⁺, let us consider the function ℎ(𝑛) defined as ℎ(𝑛)𝑥,𝑦∶=sup_Θ(𝑛)inf_𝑥(𝑛)𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]), (67) where 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛) and Θ(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛).

For 𝑘∈ℝ, the lower Peierls barrier ℎ⁻_𝑘∶𝑀×𝑁→ℝ is ℎ⁻_𝑘𝑥,𝑦∶=liminf_𝑛→∞ℎ(𝑛)𝑥,𝑦+𝑘𝑛. (68)

Proposition 8. For every 𝑘∈ℝ, one has the following: (1)if ℎ⁻_𝑘(𝑥,𝑦)=−∞ for some (𝑥,𝑦)∈𝑀×𝑁, then ℎ⁻_𝑘(𝑧,𝑤)=−∞, for all (𝑧,𝑤)∈𝑀×𝑁;(2)if ℎ⁻_𝑘(𝑥,𝑦)=+∞ for some (𝑥,𝑦)∈𝑀×𝑁, then ℎ⁻_𝑘(𝑧,𝑤)=+∞, for all (𝑧,𝑤)∈𝑀×𝑁.

Proof. Let 𝐵 be positive and consider 𝐶>sup𝑉−inf𝑉+𝐵.(1)Let us take (𝑧,𝑤)∈𝑀×𝑁 and suppose that ℎ⁻_𝑘(𝑥,𝑦)=−∞. Hence, there is 𝑛_𝑗↑∞ such that lim_𝑗→∞ℎ𝑛_𝑗𝑥,𝑦+𝑘𝑛_𝑗=−∞. (69) Therefore, there is 𝐽∈ℤ⁺ such that if 𝑗≥𝐽, then ℎ𝑛_𝑗𝑥,𝑦+𝑘𝑛_𝑗<−𝐶. (70) Let Θ(𝑛_𝑗)∈ℰ_𝑁(𝑧,𝑤,𝑛_𝑗) be a strategy. We define Θ⁰(𝑛_𝑗)∈ℰ_𝑁(𝑥,𝑦,𝑛_𝑗) as follows: for 𝑥(𝑛_𝑗)={𝑥,𝑥₁,…,𝑥_𝑛𝑗}∈𝑆_𝑀(𝑥,𝑛_𝑗), Θ⁰𝑛_𝑗𝑥𝑛_𝑗𝑖=Θ𝑛𝑦,𝑖=0,_𝑗𝑧,𝑥₁,…,𝑥_𝑛𝑗𝑖,1≤𝑖≤𝑛_𝑗. (71) Therefore, inf_{𝑥(𝑛𝑗)}𝐴^𝑛𝑗(𝑥(𝑛_𝑗),Θ⁰(𝑛_𝑗)[𝑥(𝑛_𝑗)])+𝑘𝑛_𝑗<−𝐶. Hence, there is 𝑥⁰(𝑛_𝑗)={𝑥,𝑥⁰₁,…,𝑥⁰_𝑛𝑗} such that 𝐴^𝑛𝑗𝑥⁰𝑛_𝑗,Θ⁰𝑛_𝑗𝑥⁰𝑛_𝑗+𝑘𝑛_𝑗<−𝐶. (72) If we define 𝑧⁰(𝑛_𝑗)={𝑧,𝑥⁰₁,…,𝑥⁰_𝑛𝑗}∈𝑆_𝑀(𝑧,𝑛_𝑗), then 𝐴^𝑛𝑗𝑧⁰𝑛_𝑗,Θ𝑛_𝑗𝑧⁰𝑛_𝑗+𝑘𝑛_𝑗=𝑉𝑧,𝑤,𝑥⁰₁,Θ𝑛_𝑗𝑧⁰𝑛_𝑗1−𝑉𝑥,𝑦,𝑥⁰₁,Θ𝑛_𝑗𝑧⁰𝑛_𝑗1+𝐴^𝑛𝑗𝑥⁰𝑛_𝑗,Θ⁰𝑛_𝑗𝑥⁰𝑛_𝑗+𝑘𝑛_𝑗<sup𝑉−inf𝑉−𝐶<−𝐵, (73) in which it follows that inf_{𝑧(𝑛𝑗)}𝐴^𝑛𝑗(𝑧(𝑛_𝑗),Θ(𝑛_𝑗)[𝑧(𝑛_𝑗)])+𝑘𝑛_𝑗<−𝐵, for every Θ(𝑛_𝑗)∈ℰ_𝑁(𝑧,𝑤,𝑛_𝑗), where 𝑗≥𝐽. Then, lim_𝑗→∞ℎ𝑛_𝑗(𝑧,𝑤)+𝑘𝑛_𝑗=−∞; (74) so ℎ⁻_𝑘(𝑧,𝑤)=−∞.(2) Let us take (𝑧,𝑤)∈𝑀×𝑁 and suppose that ℎ⁻_𝑘(𝑥,𝑦)=∞. There is 𝑁∈ℤ⁺ such that if 𝑛≥𝑁, then ℎ(𝑛)𝑥,𝑦+𝑘𝑛>𝐶. (75) Therefore, there is Θ⁰(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛) such that inf_𝑥(𝑛)𝐴^𝑛𝑥(𝑛),Θ⁰(𝑛)[𝑥(𝑛)]+𝑘𝑛>𝐶. (76) Taking 𝑧(𝑛)={𝑧,𝑧₁,…,𝑧_𝑛}∈𝑆_𝑀(𝑧,𝑛), define Θ¹(𝑛)∈ℰ_𝑁(𝑧,𝑤,𝑛) as Θ¹(𝑛)[𝑧(𝑛)]_𝑖=Θ𝑤,𝑖=0,₀(𝑛)𝑥,𝑧₁,…,𝑧_𝑛𝑖,1≤𝑖≤𝑛. (77) So, considering 𝑥(𝑛)={𝑥,𝑧₁,…,𝑧_𝑛}∈𝑆_𝑀(𝑥,𝑛), 𝐴^𝑛𝑧(𝑛),Θ¹(𝑛)[𝑧(𝑛)]+𝑘𝑛=𝑉𝑧,𝑤,𝑧₁,Θ¹(𝑛)[𝑧(𝑛)]₁−𝑉𝑥,𝑦,𝑧₁,Θ⁰(𝑛)[𝑥(𝑛)]₁+𝐴^𝑛𝑥(𝑛),Θ⁰(𝑛)[𝑥(𝑛)]+𝑘𝑛>inf𝑉−sup𝑉+𝐶>𝐵. (78) Since the previous relation is valid for every 𝑧(𝑛)∈𝑆_𝑀(𝑧,𝑛), it follows that inf_𝑥(𝑛)𝐴^𝑛𝑧(𝑛),Θ¹(𝑛)[𝑧(𝑛)]+𝑘𝑛>𝐵. (79) Hence, ℎ⁻_𝑘(𝑧,𝑤)=lim_𝑛→∞{ℎ(𝑛)(𝑧,𝑤)+𝑘𝑛}=∞. (80)

In order to see that ℎ⁻_𝑘(𝑥,𝑦) has a radical change on its values, for a particular value of 𝑘, we prove the following statement.

Lemma 9. There exists 𝑘>0 large enough, such that(1)ℎ⁻_−𝑘(𝑥,𝑦)=−∞. (2)ℎ⁻_𝑘(𝑥,𝑦)=+∞.

Proof. Considering 𝑘>sup𝑉+1, we have 𝑉−𝑘≤sup𝑉−𝑘<−1. If 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛) and Θ(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛), then 𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)])−𝑘𝑛<−𝑛, for each 𝑛∈ℤ⁺. Hence, ℎ⁻_−𝑘𝑥,𝑦=−∞. (81) The other relation can be obtained similarly.

It is clear that ℎ⁻_𝑘 is monotone in 𝑘.

Lemma 10. Let 𝑘,𝑘₁, and 𝑘₂ be real numbers such that 𝑘₁<𝑘<𝑘₂, then(1)if ℎ⁻_𝑘(𝑥,𝑦)=−∞, then ℎ⁻_𝑘1(𝑥,𝑦)=−∞;(2)if ℎ⁻_𝑘(𝑥,𝑦)=+∞, then ℎ⁻_𝑘2(𝑥,𝑦)=+∞;(3)moreover, if ℎ⁻_𝑘(𝑥,𝑦)∈ℝ, then ℎ⁻_𝑘1ℎ𝑥,𝑦=−∞,_−𝑘2𝑥,𝑦=+∞. (82)

Corollary 11. There exists 𝑐⁻∈ℝ, such that inf𝑘∈ℝ∶ℎ⁻_𝑘=+∞=sup𝑘∈ℝ∶ℎ⁻_𝑘=−∞=𝑐⁻. (83)

Proof. From the previous results, it follows that there are inf𝑘∈ℝ∶ℎ⁻_𝑘=+∞,sup𝑘∈ℝ∶ℎ⁻_𝑘=−∞∈ℝ. (84) Given (𝑥,𝑦)∈𝑀×𝑁, choose 𝑘₁,𝑘₂∈ℝ such that ℎ⁻_𝑘1(𝑥,𝑦)=−∞ and ℎ⁻_𝑘2(𝑥,𝑦)=+∞, then 𝑘₁≤𝑘₂. Therefore, sup{𝑘∈ℝ∶ℎ⁻_𝑘=−∞}≤𝑘₂, so sup𝑘∈ℝ∶ℎ⁻_𝑘=−∞≤inf𝑘∈ℝ∶ℎ⁻_𝑘=+∞. (85)
Suppose that sup{𝑘∈ℝ∶ℎ⁻_𝑘=−∞}<inf{𝑘∈ℝ∶ℎ⁻_𝑘=+∞}, then there are 𝑘₀∈ℝ and 𝜀>0 such that ℎ⁻_𝑘0(𝑥,𝑦)∈ℝ and sup𝑘∈ℝ∶ℎ⁻_𝑘=−∞<𝑘₀−𝜀<𝑘₀+𝜀<inf𝑘∈ℝ∶ℎ⁻_𝑘=+∞. (86) Lemma 10 shows that ℎ⁻_𝑘0−𝜀=−∞ and ℎ⁻_𝑘0+𝜀=+∞; this is a contradiction. Hence, sup𝑘∈ℝ∶ℎ⁻_𝑘=−∞=inf𝑘∈ℝ∶ℎ⁻_𝑘=+∞. (87)

Theorem 2 provides us a different characterization for the critical value 𝑐.

Proof. Let 𝑢 be a fixed point of the Lax operator with critical value 𝑐. For (𝑥,𝑦)∈𝑀×𝑁 and 𝑛∈ℤ⁺, 𝑢𝑥,𝑦+𝑛𝑐=ℒ^𝑛𝑢𝑥,𝑦=inf_Θ(𝑛)sup_𝑥(𝑛)𝑢𝑥_𝑛,Θ(𝑛)[𝑥(𝑛)]_𝑛−𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]), (88) where 𝑥(𝑛)∈𝑆_𝑀(𝑥,𝑛) and Θ(𝑛)∈ℰ_𝑁(𝑥,𝑦,𝑛). Therefore, inf𝑢+𝑛𝑐≤sup𝑢−sup_Θ(𝑛)inf_𝑥(𝑛)𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]); (89) and then, for every 𝑛∈ℤ⁺, ℎ(𝑛)𝑥,𝑦+𝑐𝑛≤sup𝑢−inf𝑢. (90) So ℎ⁻_𝑐(𝑥,𝑦)<+∞; on that account, 𝑐≤𝑐⁻. Analogously, from (88), we obtain sup𝑢+𝑛𝑐≥inf𝑢−sup_Θ(𝑛)inf_𝑥(𝑛)𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]); (91) consequently, for each 𝑛∈ℤ⁺, ℎ(𝑛)𝑥,𝑦+𝑐𝑛≥inf𝑢−sup𝑢. (92) Hence, ℎ⁻_𝑐(𝑥,𝑦)>−∞ and in addition 𝑐≥𝑐⁻.

We can introduce the upper Peierls barrier ℎ⁺_𝑘∶𝑀×𝑁→ℝ defined by ℎ⁺_𝑘𝑥,𝑦∶=limsup_𝑛→∞ℎ(𝑛)𝑥,𝑦+𝑘𝑛. (93) By similar arguments to the ones we made for ℎ⁻_𝑘, we obtain analogous properties for ℎ⁺_𝑘 and the following result.

Theorem 12. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and (𝑥,𝑦)∈𝑀×𝑁, then 𝑐=inf𝑘∈ℝ∶ℎ⁺_𝑘=+∞=sup𝑘∈ℝ∶ℎ⁺_𝑘=−∞, (94) and ℎ⁺_𝑐(𝑥,𝑦)∈ℝ, where 𝑐 is the critical value.

On account of lower and upper Peierls barrier properties, we have ℎ⁻_𝑘=ℎ⁺_𝑘 if 𝑘≠𝑐.

Corollary 13. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and (𝑥,𝑦)∈𝑀×𝑁, then sup_𝑛∈ℤ+𝑛𝑐+ℎ(𝑛)𝑥,𝑦<+∞, (95) where 𝑐 is the critical value.

Proof. There is 𝑁∈ℤ⁺ such that ℎ⁻_𝑐𝑥,𝑦−1<𝑛𝑐+ℎ(𝑛)𝑥,𝑦<ℎ⁺_𝑐𝑥,𝑦+1, (96) for 𝑛≥𝑁. Let 𝐵 be the maximum of numbers |ℎ⁻_𝑐(𝑥,𝑦)−1|, |ℎ⁺_𝑐(𝑥,𝑦)+1|, |𝑐+ℎ(1)(𝑥,𝑦)|,|2𝑐+ℎ(2)(𝑥,𝑦)|,…,|(𝑁−1)𝑐+ℎ(𝑁−1)(𝑥,𝑦)|. According to Theorems 2 and 12, we have ℎ⁻_𝑐(𝑥,𝑦),ℎ⁺_𝑐(𝑥,𝑦)∈ℝ, so 𝐵∈ℝ, and in addition, 𝑛𝑐+ℎ(𝑛)𝑥,𝑦≤𝐵, (97) for all 𝑛∈ℤ⁺.

Corollary 14. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and (𝑥,𝑦)∈𝑀×𝑁, then −𝑐=lim_𝑛→∞1𝑛ℎ(𝑛)𝑥,𝑦, (98) where 𝑐 is the critical value.

Theorem 3 states that 𝑐 can be written in terms of actions for certain special sequences, using a solution of the Hamilton-Jacobi equation.

Proof. Consider 𝑛∈ℤ⁺, (𝑥,𝑦)∈𝑀×𝑁, and 𝑢∈𝐶(𝑀×𝑁,ℝ) solutions of the Hamilton-Jacobi equation (66). Because 𝑀 and 𝑁 are compact spaces, for each 𝑛∈ℤ⁺, given 𝑥_𝑛∈𝑀, we can choose 𝑦^∗_𝑛𝑥_𝑛∈argmin_{𝑦𝑛∈𝑁}𝑢𝑥_𝑛,𝑦_𝑛−𝑢𝑥^∗_𝑛−1,𝑦^∗_𝑛−1𝑥^∗_𝑛−1−𝑉𝑥^∗_𝑛−1,𝑦^∗_𝑛−1𝑥^∗_𝑛−1,𝑥_𝑛,𝑦_𝑛, (99) where (𝑥^∗₀,𝑦^∗₀(𝑥^∗₀))=(𝑥,𝑦). We may now choose 𝑥^∗_𝑛∈argmax_{𝑥𝑛∈𝑀}𝑢𝑥_𝑛,𝑦^∗_𝑛𝑥_𝑛−𝑢𝑥^∗_𝑛−1,𝑦^∗_𝑛−1𝑥^∗_𝑛−1−𝑉𝑥^∗_𝑛−1,𝑦^∗_𝑛−1𝑥^∗_𝑛−1,𝑥_𝑛,𝑦^∗_𝑛𝑥_𝑛. (100) On account of 𝑢 is the solution of (66), we have 𝑐=𝑢𝑥^∗_𝑛,𝑦^∗_𝑛𝑥^∗_𝑛−𝑢𝑥^∗_𝑛−1,𝑦^∗_𝑛−1𝑥^∗_𝑛−1−𝑉𝑥^∗_𝑛−1,𝑦^∗_𝑛−1𝑥^∗_𝑛−1,𝑥^∗_𝑛,𝑦^∗_𝑛𝑥^∗_𝑛. (101) Consequently, for each 𝑛∈ℤ⁺, we have 𝑛𝑐=𝑢𝑥^∗_𝑛,𝑦^∗_𝑛𝑥^∗_𝑛−𝑢𝑥,𝑦−𝐴^𝑛𝑥^∗(𝑛),𝑦^∗(𝑛), (102) where 𝑥^∗(𝑛)={𝑥^∗_𝑖}^𝑛_𝑖=0∈𝑆_𝑀(𝑥,𝑛) and 𝑦^∗(𝑛)={𝑦^∗_𝑖(𝑥^∗_𝑖)}^𝑛_𝑖=0∈𝑆_𝑁(𝑦,𝑛).
By the compactness of 𝑀 and 𝑁 and because 𝑢∈𝐶(𝑀×𝑁,ℝ) we obtain that inf𝑢−sup𝑢≤𝑛𝑐+𝐴^𝑛𝑥^∗(𝑛),𝑦^∗(𝑛)≤sup𝑢−inf𝑢, (103) from which we conclude that 𝑛𝑐+𝐴^𝑛𝑥^∗(𝑛),𝑦^∗(𝑛)≤sup𝑢−inf𝑢<+∞, (104) for all 𝑛∈ℤ⁺.

We can obtain another characterization of the critical value in terms of optimal sequences in the previous theorem.

Corollary 15. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, and (𝑥,𝑦)∈𝑀×𝑁, then there are sequences {𝑥^∗_𝑖}^∞_𝑖=0⊂𝑀 and {𝑦^∗_𝑖}^∞_𝑖=0⊂𝑁 such that −𝑐=lim_𝑛→∞1𝑛𝐴^𝑛𝑥^∗(𝑛),𝑦^∗(𝑛), (105) where 𝑥^∗(𝑛)={𝑥^∗_𝑖}^𝑛_𝑖=0∈𝑆_𝑀(𝑥,𝑛) and 𝑦^∗(𝑛)={𝑦^∗_𝑖}^𝑛_𝑖=0∈𝑆_𝑁(𝑦,𝑛), where 𝑐 is the critical value.

4. Critical Solutions

Let 𝑀 and 𝑁 be compact metric spaces, and let 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ be a Lipschitz function.

Definition 16. For 𝑘∈ℝ, a function 𝑔∈𝐶(𝑀×𝑁,ℝ) is called 𝑘-supersolution if ℒ¹𝑔≥𝑔+𝑘. (106) Analogously, a function 𝑔∈𝐶(𝑀×𝑁,ℝ) is called 𝑘-subsolution if ℒ¹𝑔≤𝑔+𝑘. (107) The function 𝑔 is a 𝑘-solution, if 𝑔 is 𝑘-supersolution and 𝑘-subsolution.

It is evident that 𝑔 is a 𝑘-solution if and only if 𝑔 is a fixed point of the Lax operator with critical value 𝑘; in this case 𝑘=𝑐 according to Theorem 1. The 𝑐-solutions are called critical solutions.

Proposition 17. If 𝑀 and 𝑁 are compact metric spaces, 𝑉∶𝑀×𝑁×𝑀×𝑁→ℝ is a Lipschitz function, 𝑘∈ℝ, and (𝑥₀,𝑦₀)∈𝑀×𝑁, then (1)𝑔 is 𝑘-supersolution if and only if there exists 𝑥₁∈𝑀 such that 𝑔𝑥₁,𝑦₁−𝑔𝑥₀,𝑦₀−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≥𝑘, (108) for all 𝑦₁∈𝑁;(2)𝑔 is 𝑘-subsolution if and only if for every 𝑥₁∈𝑀, there exists 𝑦₁=𝑦₁(𝑥₁)∈𝑁 such that 𝑔𝑥₁,𝑦₁−𝑔𝑥₀,𝑦₀−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≤𝑘. (109)

Proof. Consider (𝑥₀,𝑦₀)∈𝑀×𝑁, (1)𝑔 satisfies (108) if and only if there is 𝑥₁∈𝑀 such that inf_𝑦1∈𝑁𝑔𝑥₁,𝑦₁−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≥𝑔𝑥₀,𝑦₀+𝑘, (110) so ℒ¹𝑔𝑥₀,𝑦₀=inf_Θ(1)sup_𝑥(1)𝑔𝑥₁,Θ(1)[𝑥(1)]₁−𝑉𝑥₀,𝑦₀,𝑥₁,Θ(1)[𝑥(1)]₁=sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑔𝑥₁,𝑦₁−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≥𝑔𝑥₀,𝑦₀+𝑘, (111) where 𝑥(1)∈𝑆_𝑀(𝑥₀,1) and Θ(1)∈ℰ_𝑁(𝑥₀,𝑦₀,1);(2)𝑔 satisfies (109) if and only if for every 𝑥₁∈𝑀, we have inf_𝑦1∈𝑁𝑔𝑥₁,𝑦₁−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≤𝑔𝑥₀,𝑦₀+𝑘. (112) Next, ℒ¹𝑔𝑥₀,𝑦₀=inf_Θ(1)sup_𝑥(1)𝑔𝑥₁,Θ(1)[𝑥(1)]₁−𝑉𝑥₀,𝑦₀,𝑥₁,Θ(1)[𝑥(1)]₁=sup_𝑥1∈𝑀inf_𝑦1∈𝑁𝑔𝑥₁,𝑦₁−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≤𝑔𝑥₀,𝑦₀+𝑘, (113) where 𝑥(1)∈𝑆_𝑀(𝑥₀,1) and Θ(1)∈ℰ_𝑁(𝑥₀,𝑦₀,1).

We have another consequence of the previous result: if 𝑛∈ℤ⁺, the critical solutions 𝑢 satisfy the following statements: (1)for every strategy Θ(𝑛)∈ℰ_𝑁(𝑥₀,𝑦₀,𝑛), there exists 𝑥(𝑛)∈𝑆_𝑀(𝑥₀,𝑛) such that 𝑢𝑥_𝑛,Θ(𝑛)𝑥(𝑛)_𝑛−𝑢𝑥₀,𝑦₀−𝐴^𝑛𝑥(𝑛),Θ(𝑛)𝑥(𝑛)≥𝑛𝑐; (114)(2)there exists a strategy Θ(𝑛)∈ℰ_𝑁(𝑥₀,𝑦₀,𝑛) such that 𝑢𝑥_𝑛,Θ(𝑛)[𝑥(𝑛)]_𝑛−𝑢𝑥₀,𝑦₀−𝐴^𝑛𝑥(𝑛),Θ(𝑛)[𝑥(𝑛∀)]≤𝑛𝑐,𝑥(𝑛)∈𝑆_𝑀𝑥₀,𝑛. (115)

We have some properties of 𝑘-super and 𝑘-subsolutions.

Lemma 18. For 𝑔∈𝐶(𝑀×𝑁,ℝ) and (𝑥₀,𝑦₀)∈𝑀×𝑁,(1)if 𝑔 is 𝑘-supersolution and 𝑘^′≤𝑘, then 𝑔 is 𝑘^′-supersolution;(2)if 𝑔 is 𝑘-subsolution and 𝑘^′≥𝑘, then 𝑔 is 𝑘^′-subsolution;(3)if the sets 𝑆_𝑈,𝑆_𝐿⊂ℝ are defined as 𝑆_𝑈𝑔=,𝑆𝑘∈ℝ∶𝑔is𝑘-𝑠𝑢𝑝𝑒𝑟𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛_𝐿𝑔=𝑘∈ℝ∶𝑔is𝑘-𝑠𝑢𝑏𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛, (116) then sup𝑆_𝑈(𝑔)≤inf𝑆_𝐿(𝑔).

Proof. The first and second items are clear. To prove the third claim, choose 𝑎∈𝑆_𝑈(𝑔) and 𝑏∈𝑆_𝐿(𝑔). Then, there is 𝑥₁∈𝑀 such that 𝑔𝑥₁,𝑦₁−𝑔𝑥₀,𝑦₀−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≥𝑎, (117) for all 𝑦₁∈𝑁. Because 𝑏∈𝑆_𝐿(𝑔), there is 𝑦₁=𝑦₁(𝑥₁) such that 𝑔𝑥₁,𝑦₁−𝑔𝑥₀,𝑦₀−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦₁≤𝑏. (118) Therefore, 𝑎≤𝑏, and then sup𝑆_𝑈(𝑔)≤inf𝑆_𝐿(𝑔).

By definition, if 𝑢 is a fixed point of the Lax operator ℒ^𝑛 with critical value 𝑐, then 𝑢 is a critical solution. Therefore, 𝑐∈𝑆_𝑈(𝑢)∩𝑆_𝐿(𝑢), and according to Lemma 18, we have a new characterization of the critical value 𝑐 (see Theorem 4).

We are interested in showing that the lower Peierls barrier is a critical solution. To this end, we consider the behavior of extremal sequences for lower Peierls barrier defined as follows.

Definition 19. We define a couple (𝑥(𝑛),Θ(𝑛))∈𝑆_𝑀(𝑥₀,𝑛)×ℰ_𝑁(𝑥₀,𝑦₀,𝑛) extremal for ℎ(𝑛)(𝑥₀,𝑦₀), if ℎ(𝑛)(𝑥₀,𝑦₀)=𝐴^𝑛(𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]) and satisfies the following conditions:(1)for every strategy Θ(𝑛)∈ℰ_𝑁(𝑥₀,𝑦₀,𝑛), there exists 𝑥(𝑛)∈𝑆_𝑀(𝑥₀,𝑛) such that 𝐴^𝑛𝑥(𝑛),Θ(𝑛)𝑥(𝑛)≥𝐴^𝑛𝑥(𝑛),Θ(𝑛)𝑥(𝑛); (119)(2)for every 𝑥(𝑛)∈𝑆_𝑀(𝑥₀,𝑛), 𝐴^𝑛𝑥(𝑛),Θ(𝑛)𝑥(𝑛)≤𝐴^𝑛𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]. (120)

The following result tells us that the lower Peierls barrier is a critical solution and therefore a solution of the Hamilton-Jacobi equation.

Theorem 20. −ℎ⁻_𝑐 is a critical solution.

Proof. Consider (𝑥₀,𝑦₀)∈𝑀×𝑁. According to Theorem 2, ℎ⁻_𝑐∈ℝ.
(I) We begin by proving −ℎ⁻_𝑐 is 𝑐-supersolution. Let us take 𝑦₁∈𝑁 and let Φ(𝑛−1)∈ℰ_𝑁(𝑥₁,𝑦₁,𝑛−1) be extremal for ℎ(𝑛−1)(𝑥₁,𝑦₁), then ℎ(𝑛−1)𝑥₁,𝑦₁=inf_{𝑧()𝑛−1}𝐴^𝑛−1𝑧(𝑛−1),Φ(𝑛−1)[𝑧(𝑛−1)], (121) where 𝑧(𝑛−1)∈𝑆_𝑀(𝑥₁,𝑛−1).
For every 𝑥(𝑛)∈𝑆_𝑀(𝑥₀,𝑛), define Θ(𝑛)∈ℰ_𝑁(𝑥₀,𝑦₀,𝑛) as Θ(𝑛)[𝑥(𝑛)]_𝑖=𝑦₀Φ(𝑥,𝑖=0,𝑛−1)₁,𝑥₂,…,𝑥_𝑛𝑖−1,1≤𝑖≤𝑛, (122) then ℎ⁻_𝑐𝑥₀,𝑦₀≥liminf_𝑛→∞inf_𝑥(𝑛)𝐴^𝑛𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]+𝑐𝑛. (123) There is 𝑥(𝑛)∈𝑆_𝑀(𝑥₀,𝑛) such that inf_𝑥(𝑛)𝐴^𝑛𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]=𝐴^𝑛𝑥(𝑛),Θ(𝑛)𝑥(𝑛). (124) Choose a subsequence 𝑛_𝑗↑∞ such that 𝑛₁=1 and liminf_𝑛→∞𝐴^𝑛𝑥(𝑛),Θ(𝑛)𝑥(𝑛)+𝑛𝑐=lim_𝑗→∞𝐴^𝑛𝑗𝑥𝑛_𝑗,Θ𝑛_𝑗𝑥𝑛_𝑗+𝑛_𝑗𝑐; (125) extracting a subsequence if necessary, we can assume that 𝑥𝑛_𝑗⟶𝑥^∗∈𝑆_𝑀𝑥₀,∞, (126) where 𝑆_𝑀(𝑥₀,∞) denotes the set of infinite sequences in 𝑀, with initial condition 𝑥₀. Therefore, ℎ⁻_𝑐𝑥₀,𝑦₀≥lim_𝑗→∞𝑉𝑥₀,𝑦₀,𝑥^∗₁,𝑦₁+𝑉𝑥+𝑐₀,𝑦₀,𝑥𝑛_𝑗1,Θ𝑛_𝑗𝑥𝑛_𝑗1−𝑉𝑥₀,𝑦₀,𝑥^∗₁,𝑦₁+𝑉𝑥𝑛_𝑗1,Θ𝑛_𝑗𝑥𝑛_𝑗1,𝑥𝑛_𝑗2,Θ𝑛_𝑗𝑥𝑛_𝑗2−𝑉𝑥^∗₁,𝑦₁,𝑥𝑛_𝑗2,Θ𝑛_𝑗𝑥𝑛_𝑗2+𝐴^𝑛𝑗−1𝑧𝑛_𝑗−1,Φ𝑛_𝑗−1𝑧𝑛_𝑗+𝑛−1_𝑗−1𝑐, (127) where 𝑧(𝑛_𝑗−1)={𝑥^∗₁,𝑥₂,…,𝑥_𝑛𝑗}. Because 𝑉 is continuous, then ℎ⁻_𝑐𝑥₀,𝑦₀≥𝑉𝑥₀,𝑦₀,𝑥^∗₁,𝑦₁+𝑐+liminf_𝑛→∞inf_{𝑧()𝑛−1}𝐴^𝑛−1𝑧(),Φ(𝑛−1𝑛−1)𝑧(+()𝑐𝑥𝑛−1)𝑛−1=𝑉₀,𝑦₀,𝑥^∗₁,𝑦₁+𝑐+ℎ⁻_𝑐𝑥^∗₁,𝑦₁, (128) which proves that there is 𝑥^∗₁∈𝑀 such that −ℎ⁻_𝑐𝑥^∗₁,𝑦₁+ℎ⁻_𝑐𝑥₀,𝑦₀−𝑉𝑥₀,𝑦₀,𝑥^∗₁,𝑦₁≥𝑐, (129) for all 𝑦₁∈𝑁. Hence, −ℎ⁻_𝑐 is 𝑐-supersolution.
(II) We proceed to show that −ℎ⁻_𝑐 is 𝑐-subsolution. Let us take 𝑥₁∈𝑀. For each 𝑛∈ℤ⁺, let Θ(𝑛)∈ℰ_𝑁(𝑥₀,𝑦₀,𝑛) be extremal for ℎ(𝑛)(𝑥₀,𝑦₀), then ℎ(𝑛)𝑥₀,𝑦₀=inf_𝑥(𝑛)𝐴^𝑛𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]. (130) Observe that Θ(𝑛)[𝑥(𝑛)]₁ depends only on 𝑥₁, because Θ(𝑛)∈ℰ_𝑁(𝑥₀,𝑦₀,𝑛). Suppose that Θ(𝑛)[𝑥(𝑛)]₁→𝑦^∗₁(𝑥₁).
For 𝑤(𝑛−1)={𝑥₁,𝑤₂,…,𝑤_𝑛}∈𝑆_𝑀(𝑥₁,𝑛−1), we can define a strategy Φ(𝑛−1)∈ℰ_𝑁(𝑥₁,𝑦^∗₁(𝑥₁),𝑛−1) as follows: Φ(𝑛−1)[𝑤(𝑛−1)]_𝑖=𝑦^∗₁𝑥₁Θ(𝑛,𝑖=0,)[𝑧(𝑛)]_𝑖+1,1≤𝑖≤𝑛−1, (131) where 𝑧(𝑛)={𝑥₀,𝑥₁,𝑤₂,𝑤₃,…,𝑤_𝑛}∈𝑆_𝑀(𝑥₀,𝑛). So ℎ(𝑛−1)𝑥₁,𝑦^∗₁𝑥₁≥inf_{𝑤()𝑛−1}𝐴^𝑛−1𝑤(𝑛−1),Φ(𝑛−1)[𝑤(𝑛−1)]. (132) We can choose 𝑤(𝑛−1)={𝑥₁,𝑤₂,…,𝑤_𝑛}∈𝑆_𝑀(𝑥₁,𝑛−1) extremal for inf_{𝑤()𝑛−1}𝐴^𝑛−1𝑤(𝑛−1),Φ(𝑛−1)[𝑤(𝑛−1)], (133) and a subsequence 𝑛_𝑗↑∞ such that ℎ⁻_𝑐𝑥₁,𝑦^∗₁𝑥₁≥lim_𝑗→∞𝐴^𝑛𝑗−1𝑤𝑛_𝑗−1,Φ𝑛_𝑗−1𝑤𝑛_𝑗𝑛−1+𝑐_𝑗−1. (134) If 𝑧(𝑛_𝑗)={𝑥₀,𝑥₁,𝑤₂,𝑤₃,…,𝑤_𝑛𝑗}, then ℎ⁻_𝑐𝑥₀,𝑦₀=liminf_𝑛→∞inf_𝑥(𝑛)𝐴^𝑛𝑥(𝑛),Θ(𝑛)[𝑥(𝑛)]+𝑛𝑐≤lim_𝑗→∞𝑉𝑥₀,𝑦₀,𝑥₁,𝑦^∗₁𝑥₁+𝑉𝑥+𝑐₀,𝑦₀,𝑥₁,Θ𝑛_𝑗𝑧𝑛_𝑗1−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦^∗₁𝑥₁+𝑉𝑥₁,Θ𝑛_𝑗𝑧𝑛_𝑗1,𝑧𝑛_𝑗2,Θ𝑛_𝑗𝑧𝑛_𝑗2−𝑉𝑥₁,𝑦^∗₁𝑥₁,𝑧𝑛_𝑗2,Θ𝑛_𝑗𝑧𝑛_𝑗2+𝐴^𝑛𝑗−1𝑤𝑛_𝑗,Φ𝑛−1_𝑗−1𝑤𝑛_𝑗+𝑛−1_𝑗−1𝑐. (135) According to the above relations and the continuity of 𝑉, if 𝑥₁∈𝑀, there is 𝑦^∗₁(𝑥₁)∈𝑁 such that −ℎ⁻_𝑐𝑥₁,𝑦^∗₁𝑥₁+ℎ⁻_𝑐𝑥₀,𝑦₀−𝑉𝑥₀,𝑦₀,𝑥₁,𝑦^∗₁𝑥₁≤𝑐. (136) Therefore, −ℎ⁻_𝑐 is a 𝑐-subsolution.