Discrete Dynamics in Nature and Society

Volume 2013 (2013), Article ID 769368, 12 pages

http://dx.doi.org/10.1155/2013/769368

## Weak KAM Solutions of a Discrete-Time Hamilton-Jacobi Equation in a Minimax Framework

Facultad de Matemáticas, UV Zona Universitaria, 91090 Xalapa, Ver, Mexico

Received 28 January 2013; Accepted 31 May 2013

Academic Editor: Stepan Agop Tersian

Copyright © 2013 Porfirio Toledo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The purpose of this paper is to study the existence of solutions of a Hamilton-Jacobi equation in a minimax discrete-time case and to show different characterizations for a real number called the critical value, which plays a central role in this work. We study the behavior of solutions of this problem using tools of game theory to obtain a “fixed point” of the Lax operator associated, considering some facts of weak KAM theory to interpret these solutions as discrete viscosity solutions. These solutions represent the optimal payoff of a zero-sum game of two players, with increasingly long time payoffs. The developed techniques allow us to study the behavior of an infinite time game without using discount factors or average actions.

#### 1. Introduction

The Hamilton-Jacobi equation is a very important tool for the study of Lagrangian systems and control theory. In the case of convex Lagrangians, Fathi, Mather, and Mañe have perfected several techniques to understand the solutions of this equation ([1–7]). These techniques can be translated to a discrete case where, instead of a Lagrangian, there is a function of two variables in the configuration space.

The aim of this paper is to study a discrete-time system that emulates a convex and concave Lagrangian. We need to study the behavior of solutions of a discrete-time version of a Hamilton-Jacobi equation in the context of game theory. A real number, called the critical value, plays a central role in this work; this number is the asymptotic average action of optimal trajectories. We study the existence and characterization of solutions of a Hamilton-Jacobi equation for this kind of game. A relevant aspect is that these techniques allow us to study infinite-time games without using discount factors or average actions.

Consider the compact metric spaces and , which are the state spaces for players and , respectively. For , a continuous function , and a Lipschitz function , we define the static game payoff by The function is the amount that player obtains from and vice versa; so player wants to maximize the payoff , while player wants to minimize it. The lower game value is defined by It is observed in [8] that it is possible to express the lower game value as follows: where the function is called a strategy for player . These functions are very important in this work and will be defined in general for a zero-sum game with finite horizon.

For , consider the finite horizon problem, where players and alternate their moves beginning at initial state and setting with them the state trajectories determined by finite sequences and , denoted by and , respectively. The state of the system depends on the initial condition and the players decisions to get from to , where . For , we will denote by the sets of finite sequences in and , with initial states and . We define the action by and the game payoff by

Then, the lower game value with finite horizon and initial state is However, can be defined directly using progressive strategies (see [8–10]). The idea is to use a tool that allows player to select one point knowing the current and past opponent choices, without knowing the future choices. To this end, we define a progressive strategy for player as a function which satisfies the following property: for each such that and , we have where is the th element of sequence . The set of progressive strategies for player is denoted by Finally, the lower game value can be expressed as where and . This function is called Lax operator and will be essential for understanding the asymptotic behavior of game payoffs. The Proposition 6 shows that

In the next section of this paper, the objective is to prove the existence of a kind of “fixed point” of the Lax operator. We will adapt the methods developed in the work of Fathi ([3]) about weak KAM theory, now in a minimax context of game theory. It will be necessary to prove that the Lax operator , defined as in (10), satisfies some important properties: the semigroup property that is also call Dynamic Programming Principle (see [8–10]), a regularity property, monotony, and weak contraction.

Theorem 1. *If and are compact metric spaces and is a Lipschitz function, then there exist a Lipschitz function and a unique constant , such that
*

The functions in the previous result are called fixed points of the Lax operator with critical value . These fixed points are solutions of the following discrete-time Hamilton-Jacobi equation associated to : See [11] for a minimal discrete-time case.

In the third section, we will consider an infinite horizon problem, which will help us to find a different characterization of critical value . To this end, we define the lower Peierls barrier as follows, given a real number and an initial state : where and , see [3, 12, 13].

It is possible to prove that there exists a number , such that where the value of does not depend on . The critical value has a new interpretation characterized by the following result.

Theorem 2. *If and are compact metric spaces, is a Lipschitz function, and , then and , where is the critical value.*

Analogously, we can define the upper Peierls barrier by for which we can obtain similar results.

Considering that and , for every , we prove that and therefore,

In addition, using a solution of the Hamilton-Jacobi equation, we can find an optimal strategy, from which optimal sequences are obtained for both players for all times. In particular, we prove the following result.

Theorem 3. *If and are compact metric spaces, is a Lipschitz function, and , then there are sequences and such that
**
where and , where is the critical value.*

From this, it follows immediately that

The sequences obtained in the previous result satisfy the Bellman principle of optimality, because from each state in the optimal trajectory, the complement remains optimal. The optimality of the remaining decisions from depends on this last state, which is the result of the previous decisions.

In last section, we will show another characterization for the solutions of the Hamilton-Jacobi equation.

For , a function is called -supersolution if Analogously, a function is called -subsolution if The function is a -solution if is -supersolution and -subsolution (see [4]).

Clearly, is a -solution if and only if is a fixed point of the Lax operator with critical value . According to Theorem 1, if there exists a unique constant for fixed points of the Lax operator, then there is only one kind of -solutions, in fact for . The -solutions are called critical solutions. From this, we get another characterization of , which appears in the following result.

Theorem 4. *If and are compact metric spaces, is a Lipschitz function, and is a fixed point of the Lax operator with critical value , then
**
where
*

Finally, we will show that is a critical solution and therefore a solution of the Hamilton-Jacobi equation.

#### 2. Lax Operator

Let and be compact metric spaces, a Lipschitz function, and . Consider a zero-sum game of two players, and , with finite horizon , where and represent the state spaces and sequences and determine the state trajectories, respectively, for players and . The function , defined by is the game action. For and , the game payoff is defined by If is the set of progressive strategies for player , the Lax operator is the lower game value, defined by with and . We prove the following regularity property for the Lax operator.

Lemma 5. *The Lax operator is -Lipschitz, for any and , where is the Lipschitz constant of .*

*Proof. *We will consider , , and . There exists such that
where .

For a finite sequence , we define as

For this strategy, we have
where . It follows that there is such that

We define as . Combining (28) with (31), we obtain
Since and , for , we have
where is the Lipschitz constant of and is a distance in . Similarly, we obtain the opposite inequality. Thus,

From above result, we prove the semigroup property for the Lax operator, also called the Principle of Dynamic Programming.

Proposition 6 (Semigroup Property). *For ,
**
where and .*

*Proof. *We will consider and . To prove the relation , we will see that, for every , the inequalities and take place simultaneously.

(I) Let us first prove that . To do this, take such that
where .

On the other hand, for each ,
where and . Hence, there is such that

We will define a using the previous strategy and the family . Given a sequence in , the associated sequence in is defined as follows: the fist terms are given by the strategy and the second part is given by , for a specific . More precisely, for ,
where , , , and . Taking any and combining (36) with (38), we obtain
Therefore,
with . This implies that
and consequently .

(II) It remains to prove that . To this end, we will consider such that
where .

We will define as the -time restriction of . Given a sequence , the associated sequence in is defined as follows: the terms are given by the fist terms of strategy applied to some extension of in , adding any sequence . Since is a strategy, it follows that the function is well defined and it is a strategy. More precisely, for , taking any , we define as
for , where . So
where . Consequently, there is such that
Once that is fixed so will be . We now define , for , as
for , where . We have
where . Therefore, there is such that
Define as
Combining (43) and (46) with (49) we obtain:
for this reason .

Lemma 7. *If , then*(1)*(monotony) if , then ;*(2)*if , then ;*(3)*(weak contraction) .*

*Proof. *The first two properties are a direct consequence of the definition. For last the claim, choose , then
The previous items imply that
So , for every . As a result,

Theorem 1 is analogue to Weak KAM Theorem. We use the same argument of Fathi (see [3]) to prove the existence of fixed points of this discrete-time case of the Lax operator.

According to this result, we say that is a fixed point of the Lax operator with critical value .

*Proof. *(I) To prove the existence of a fixed point, we will use the same argument used by Fathi ([3]) for the continuous-time case.

Let be the quotient of the set of continuous functions modulo constants, . The norm is defined as
for . Then, is a Banach space. We define the operator as
which satisfies the weak contraction property. Thus, , with , is a contraction; so there are fixed points of . As is equi-Lipschitz, since Arzela-Ascoli theorem, it follows that has a uniformly convergent subsequence when . Then, has a fixed point , which in terms of means that there is a constant such that
for each positive integer . From the semigroup property, there is such that
for all positive integers .

Function is a Lipschitz function, because is also a Lipschitz function. In fact, according to Lemma 5, is -Lipschitz, where is the Lipschitz constant of .

(II) It remains to prove that the number is unique. Let be positive. Suppose there are and , , such that , with . Choose such that
For this strategy, we also have that
So there is , such that
Therefore, using the hypothesis, we obtain
Considering the subtraction of the previous inequalities, we obtain following relations:
then
On the other hand, because , we have when , which is a contradiction.

Considering the one-step case (see [8]), we can rewrite the Lax operator as If is a fixed point with critical value , then is a solution for the following discrete-time analog of the Hamilton-Jacobi equation:

Conversely, if is a solution of the previous Hamilton-Jacobi equation, then is a fixed point of the Lax operator with critical value . According to semigroup property and relation in Lemma 7, it follows that is a fixed point of with critical value , for all .

#### 3. Peierls Barrier

Let us now study an infinite horizon problem for players and . Let and be compact metric spaces, and let be a Lipschitz function. For , let us consider the function defined as where and .

For , the lower Peierls barrier is

Proposition 8. *For every , one has the following: *(1)if for some , then , for all ;(2)if for some , then , for all .

*Proof. *Let be positive and consider .(1)Let us take and suppose that . Hence, there is such that
Therefore, there is such that if , then
Let be a strategy. We define as follows: for ,
Therefore, . Hence, there is such that
If we define , then
in which it follows that , for every , where . Then,
so .(2) Let us take and suppose that . There is such that if , then
Therefore, there is such that
Taking , define as
So, considering ,
Since the previous relation is valid for every , it follows that
Hence,

In order to see that has a radical change on its values, for a particular value of , we prove the following statement.

Lemma 9. *There exists large enough, such that*(1)*. *(2)*. *

*Proof. *Considering , we have . If and , then , for each . Hence,
The other relation can be obtained similarly.

It is clear that is monotone in .

Lemma 10. *Let , and be real numbers such that , then*(1)*if , then ;*(2)*if , then ;*(3)*moreover, if , then
*

Corollary 11. *There exists , such that
*

*Proof. *From the previous results, it follows that there are
Given , choose such that and , then . Therefore, , so

Suppose that , then there are and such that and
Lemma 10 shows that and ; this is a contradiction. Hence,

Theorem 2 provides us a different characterization for the critical value .

*Proof. *Let be a fixed point of the Lax operator with critical value . For and ,
where and . Therefore,
and then, for every ,
So ; on that account, . Analogously, from (88), we obtain
consequently, for each ,
Hence, and in addition .

We can introduce the upper Peierls barrier defined by By similar arguments to the ones we made for , we obtain analogous properties for and the following result.

Theorem 12. *If and are compact metric spaces, is a Lipschitz function, and , then
**
and , where is the critical value.*

On account of lower and upper Peierls barrier properties, we have if .

Corollary 13. *If and are compact metric spaces, is a Lipschitz function, and , then
**
where is the critical value.*

*Proof. *There is such that
for . Let be the maximum of numbers , , . According to Theorems 2 and 12, we have , so , and in addition,
for all .

Corollary 14. *If and are compact metric spaces, is a Lipschitz function, and , then
**
where is the critical value.*

Theorem 3 states that can be written in terms of actions for certain special sequences, using a solution of the Hamilton-Jacobi equation.

*Proof. *Consider , , and solutions of the Hamilton-Jacobi equation (66). Because and are compact spaces, for each , given , we can choose
where . We may now choose