#### Abstract

We investigate the global dynamics of several anticompetitive systems of rational difference equations which are special cases of general linear fractional system of the forms ., where all parameters and the initial conditions are arbitrary nonnegative numbers, such that both denominators are positive. We find the basins of attraction of all attractors of these systems.

#### 1. Introduction

A first order system of difference equations:
where , are continuous functions, is *competitive* if is nondecreasing in and nonincreasing in , and is nonincreasing in and nondecreasing in .

System (1) where the functions and have monotonic character opposite the monotonic character in competitive system is called *anticompetitive*; see [1, 2].

In this paper, we consider the following anticompetitive systems of difference equations: where all parameters are positive numbers and the initial conditions are arbitrary nonnegative numbers such that . In the classification of all linear fractional systems in [3], System (2) was mentioned as system . We also consider systems with , , which were labeled as systems and , respectively, in [3]. Three systems have interesting and different dynamics. While System (2) has all bounded solutions, most of solutions of Systems (3) and (4) are unbounded. Another major difference is the existence of the unique period-two solution for (2) and, in a special case, the abundance of such solutions, while neither (3) nor (4) has period-two solutions. We show that every solution of System (3) converges to the unique equilibrium or is approaching and so System (3) gives an example of a semistable equilibrium point. Finally, we show that all solutions of (4) which start on the stable set converge to the unique equilibrium, while all solutions which start off the stable set are approaching or . We also get that for some special values of parameters, Systems (4) and (3) can be decoupled and explicitly solved.

Competitive systems of the form (1) were studied by many authors such as Clark and Kulenović [4], Hess [5], Hirsch and Smith [6], Kulenović and Merino [7], Kulenović and Nurkanović [8], Garić-Demirović et al. [9, 10], and Smith [11, 12]. Precise results about the basins of attraction of the equilibrium points have been obtained in [13].

The study of anticompetitive systems started in [1] and has advanced since then; see [2]. The principal tool of study of anticompetitive systems is the fact that the second iterate of the map associated with anticompetitive system is competitive map and so elaborate theory for such maps developed recently in [6, 14, 15] can be applied.

The major result on a global behavior of System (2) is the following theorem.

Theorem 1. *(a) Assume that . Then the unique positive equilibrium point of System (2) is locally asymptotically stable with the basin of attraction . The unique period-two solution is a saddle point and its basin of attraction is the union of coordinate axes without the origin, that is, .**(b) Assume that . Then every solution of System (2) converges to the period-two solution or to the equilibrium . More precisely, there exists a set which is the basin of attraction of . The set has the property that for
**
the following holds.**(i) If , then
**(ii) If ,
**(c) Assume that . Then every solution of System (2) is equal to either the period-two solution , or to the equilibrium , where and .*

#### 2. Preliminaries

We now give some basic notions about competitive systems and maps in the plane of the form (1), where and are continuous functions, is nondecreasing in and nonincreasing in , and is nonincreasing in and nondecreasing in in some domain with nonempty interior.

Consider a map on a set , and let . The point is called a *fixed point* if . An *isolated* fixed point is a fixed point that has a neighborhood with no other fixed points in it. A fixed point is an *attractor* if there exists a neighborhood of such that as for ; the *basin of attraction* is the set of all such that as . A fixed point is a global attractor on a set if is an attractor and is a subset of the basin of attraction of . If is differentiable at a fixed point and if the Jacobian has one eigenvalue with modulus less than one and a second eigenvalue with modulus greater than one, is said to be a *saddle*. See [16] for additional definitions.

*Definition 2. *Let be a continuously differentiable vector function and let be a neighborhood of a saddle point of (1). The local stable manifold is the set
The global stable manifold of a saddle point is the set

The map may be viewed as a monotone map if we define a partial order on so that the positive cone in this new partial order is the fourth (resp. first) quadrant. Define a south-east (resp. north-east) partial order (resp. ) on so that the positive cone is the fourth quadrant (resp. first quadrant), that is, (resp. ) if and only if and (resp. and ). For the *order interval * is the set of all such that . The map is called competitive (resp. cooperative) on a set if (resp. ) implies (resp. ).

Two points are said to be *related* if or . Also, a strict inequality between points may be defined as if and . A stronger inequality may be defined as if and . A map is *strongly monotone* if implies that for all . Clearly, being related is invariant under the iteration of a strongly monotone map. Differentiable strongly monotone maps have Jacobian with constant sign configuration
The mean value theorem and the convexity of may be used to show that is monotone, as in [17].

For , define for to be the usual four quadrants based at and numbered in a counterclockwise direction, for example, . We now state three results for competitive maps in the plane.

The following definition is from [11].

*Definition 3. *Let be a nonempty subset of . A competitive map is said to satisfy condition () if for every , in , implies , and is said to satisfy condition () if for every , in , implies .

The following theorem was proved by de Mottoni-Schiaffino for the Poincaré map of a periodic competitive Lotka-Volterra system of differential equations. Smith generalized the proof to competitive and cooperative maps [11, 12].

Theorem 4. *Let be a nonempty subset of . If is a competitive map for which () holds, then for all , is eventually componentwise monotone. If the orbit of has compact closure, then it converges to a fixed point of . If instead () holds, then for all , is eventually componentwise monotone. If the orbit of has compact closure in , then its omega limit set is either a period-two orbit or a fixed point.*

The following result is from [11], with the domain of the map specialized to be the cartesian product of intervals of real numbers. It gives a sufficient condition for conditions () and ().

Theorem 5. *Let be the cartesian product of two intervals in . Let be a competitive map. If is injective and for all , then satisfies (). If is injective and for all , then satisfies ().*

The following result is a direct consequence of the Trichotomy Theorem of Dancer and Hess, see [5, 15, 18], and is helpful for determining the basins of attraction of the equilibrium points.

Corollary 6. *If the nonnegative cone with respect to the partial order is a generalized quadrant in and if the competitive map has no fixed points in other than and , then the interior of is either a subset of the basin of attraction of or a subset of the basin of attraction of .*

The next two results are from [15, 18].

Theorem 7. *Let be a competitive map on a rectangular region . Let be a fixed point of such that is nonempty (i.e., is not the NW or SE vertex of , and is strongly competitive on . Suppose that the following statements are true.*(a)*The map has a extension to a neighborhood of .*(b)*The Jacobian matrix of at has real eigenvalues , such that , where , and the eigenspace associated with is not a coordinate axis.**Then there exists a curve through that is invariant and a subset of the basin of attraction of , such that is tangential to the eigenspace at , and is the graph of a strictly increasing continuous function of the first coordinate on an interval. Any endpoints of in the interior of are either fixed points or minimal period-two points. In the latter case, the set of endpoints of is a minimal period-two orbit of .*

Theorem 8. *Let be intervals in with endpoints and with endpoints, respectively, with and , where and . Let be a competitive map on a rectangle and . Suppose that the following hypotheses are satisfied.*(1)* and is strongly competitive on .*(2)*The point is the only fixed point of in .*(3)*The map is continuously differentiable in a neighborhood of , and is the saddle point.*(4)*At least one of the following statements is true.(a) has no minimal period-two orbits in .(b) and only for .*

*Then the following statements are true.*(i)

*The stable manifold is connected and it is the graph of a continuous increasing curve with endpoints in . is divided by the closure of into two invariant connected regions (“below the stable set”), and (“above the stable set”), where*(ii)

*The unstable manifold is connected and it is the graph of a continuous decreasing curve.*(iii)

*For every , eventually enters the interior of the invariant set , and for every , eventually enters the interior of the invariant set .*(iv)

*Let and be the endpoints of , where . For every and every such that , there exists such that , and for every and every such that , there exists such that .*

#### 3. Global Dynamics of System (2) (See Figure 1)

The equilibrium point of System (2) satisfies the following system of equations:

It is easy to see that System (12) has unique equilibrium point in the first quadrant, for all values of the parameters. Indeed, the positive equilibrium point is an intersection of the following two curves: It is clear that at the point of intersection curve (13) is steeper than curve (14), that is, which gives This inequality is equivalent to the following inequality: which is always satisfied.

##### 3.1. Linearized Stability Analysis of System (2)

In this section we present the linearized stability analysis of the equilibrium of System (2).

Theorem 9. *(i) If , then is locally asymptotically stable.**(ii) If , then is a nonhyperbolic equilibrium point.**(iii) If , then is a saddle point.*

*Proof. *The map associated to System (2) is
The Jacobian matrix of map (18) is
and evaluated at the equilibrium point is
The characteristic equation has the following form:
and the characteristic roots are
A straightforward calculation shows that the conditions , , and , respectively, are equivalent to the conditions
On the other hand, by dividing two equilibrium equations (12) we obtain
which implies that the condition (23) is equivalent to the conditions
which completes the proof of the theorem.

##### 3.2. Global Results for System (2)

In this section we present the proof of Theorem 1 on the global dynamics of System (2). First, we prove the following result on existence and local behavior of the period-two solutions.

Lemma 10. *System (2) has the minimal period-two solution
**
for all values of parameters. If , then the solution (26) is the unique period-two solution and when there are infinitely many period-two solutions. The set
**
is a subset of the basin of attraction of the solution (26). The period-two solution (26) is locally stable if and a saddle point if . Finally the period-two solution (26) is nonhyperbolic if .*

*Proof. *The second iterate of the map is given asA period-two solution of System (2) satisfies , which immediately leads to the following equations:
which have either unique solution if or it has infinitely many solutions if . In the first case, (29) gives immediately (13) and (30) gives (14), which means that in this case the only minimal period-two solution is (26).

A straightforward calculation shows that , which shows that is a minimal period-two solution. Moreover, , for every , , which shows that the set is a subset of the basin of attraction of . The Jacobian matrix of is
The Jacobian matrix of evaluated at is
and the Jacobian matrix of evaluated at is
In both cases, the eigenvalues of the Jacobian matrix of are , , which implies the result on local stability of the minimal period-two solution .

*Proof of Theorem 1. *First, observe that the rectangle is an invariant and attracting set for the map and so is for the map . More precisely, for . The map is a competitive map on .

*Case a. *Assume that . Then in view of Theorem 9 and Lemma 10, the map has three equilibrium points , , and , where . The equilibrium points and are saddle points and is a local attractor. The ordered intervals and are both invariant sets of and in view of Corollary 6, their interiors are attracted to . If we take the point , we can find the points and , such that . Consequently, since is competitive for and so , which by continuity of implies that
and so .

*Case b. *Assume that . Then in view of Theorem 9 and Lemma 10, the map has three equilibrium points and which are local attractors and which is a saddle point. The ordered intervals and are both invariant sets for and in view of Corollary 6, their interiors are attracted to and , respectively. In view of Theorems 7 and 8, there is the set with described properties. Direct calculation shows that the half-line , is an invariant set which in view of a uniqueness of stable manifold implies that this half-line is exactly stable manifold mentioned in Theorems 7 and 8. It should be observed that because of the fact that one of the characteristic values at the equilibrium point is , this equilibrium is superattractive, that is, , for every .

*Case c. *Assume that . Then by dividing two equations of System (2) we obtain that the solution of (2) satisfies
This means that satisfies first order difference equation , where . All nonconstant solutions of are period-two solutions . Thus . In this case, System (2) becomes
which completes the proof of Case (c).

*Remark 11. *System (2) is an example of the homogeneous system which is a special case of a general System (1), where both functions and are homogeneous functions of the same degree , that is, , for all , in intersection of domains of and and all . In that case, the ratio of every solution of (1) satisfies the first order difference equation:
whose analysis gives valuable information about the dynamics of System (1), but does not provide the global dynamics. In particular, this approach cannot determine precisely the basins of attraction of different types of attractors such as equilibrium points and periodic solutions. This approach in the case of the system of linear fractional equations:
where all parameters and the initial conditions are arbitrary nonnegative numbers such that , , was first used in [19] and was systematically developed in the recent paper [20]. In [20], the authors studied all possible homogeneous systems of the form (38) and they proved that every bounded solution converges to either an equilibrium solution or to period-two solution. They were able to find a part of the basin of attraction of the period-two solution but not the complete basin of attraction. In the case of system (2) the auxiliary equation for is
Since is decreasing, every solution of the auxiliary equation is approaching not necessarily minimal period-two solution. Further analysis can be continued either by checking negative feedback condition for or by using Theorem 3.2 from [20]. In neither case the complete description of the basins of attraction of the equilibrium and the period-two solution is possible. We prefer our approach because it is more precise and also apply equally well to anticompetitive systems which are not homogeneous. The approach which is making use of homogeneous properties of functions is applicable also to the systems which are neither competitive nor anticompetitive.

#### 4. Global Dynamics of System (3) (See Figure 2)

The equilibrium point of System (3) satisfies the following system of equations: It is easy to see that System (40) has the unique equilibrium point in the first quadrant, for all values of the parameters. Indeed, the positive equilibrium point is an intersection of the following two curves: It is clear that at the point of intersection curve (41) is steeper than curve (42), that is, which gives This inequality is equivalent to the following inequality: which in turn is equivalent to

##### 4.1. Linearized Stability Analysis of System (3)

In this section we prove the following result.

Theorem 12. *The unique equilibrium of System (3) is a saddle point.*

*Proof. *The map associated to System (3) is
The Jacobian matrix of map (47) is
and evaluated at the equilibrium point is

The characteristic equation has the following form:
Set
Then the necessary and sufficient condition for (50) to have one root inside the unit circle and one root outside the unit circle is , ; see [7, 21]. The condition leads to which is equivalent to
which is condition (46).

The condition becomes
which is equivalent to
which is clearly satisfied.

##### 4.2. Global Results for System (3)

Lemma 13. *System (3) has no minimal period-two solution. *

*Proof. *The second iterate of the map is given as
Period-two solution satisfies which reduces to the following two equations:
Equation (57) leads immediately to which is exactly the equilibrium equation (41). Using (41) in (56), we obtain after some elementary simplification that period-two solution satisfies (42). This shows that System (3) has no minimal period-two solution.

Lemma 14. *The maps and associated with System (3) have the following properties. *(i)*The maps and are injective.*(ii)* for all , .**Consequently, satisfies condition and so is eventually componentwise monotone.*

*Proof. *(i) We will prove that is injective and the injectivity of will follow immediately. The condition
is reduced to the following two conditions:
which immediately implies and so .

(ii) A direct calculation shows that
which implies our statement.

The statement on condition follows from Theorem 5.

Theorem 15. *Consider System (3). Then there exists a set which is invariant subset of the basin of attraction of . The set is a graph of a strictly increasing continuous function of the first variable on an interval and separates into two connected and invariant components, namely,
**
which satisfy the following.**(i) If , then
**(ii) If ,
*

*Proof. *Clearly, the rectangle is an invariant and attracting set for the map . In particular, for .

Then by Lemma 14 and Theorem 4, every solution of System (3) has eventually monotone components and , which shows that every bounded solution converges to period-two solution. In view of Lemma 13, there are no minimal period-two solutions and so every bounded solution of System (3) converges to the equilibrium . In view of Theorems 7 and 8, there is a set with described properties. Consequently, every solution with an initial point converges to , while every solution which starts in approaches and is asymptotic to the global unstable manifold .

#### 5. Global Dynamics of System (4) (See Figure 3)

The equilibrium point of System (4) satisfies the following system of equations: It is easy to see that System (64) has the unique equilibrium point in the first quadrant, which is an intersection of two parabolas: It is clear that at the point of intersection , curve (65) is steeper than curve (66), that is, which gives

##### 5.1. Linearized Stability Analysis of System (4)

In this section we prove the following result.

Theorem 16. *The unique equilibrium of System (4) is a saddle point. *

*Proof. *The map associated to System (4) is
The Jacobian matrix of map (69) has the form:
which evaluated at the equilibrium point is
The characteristic equation of System (4) has the following form:
Set
The necessary and sufficient condition for (72) to have one root inside the unit circle and one root outside the unit circle is , ; see [7, 21]. In view of the fact that , the condition leads to which is equivalent to
which is equivalent to the condition (68).

The condition becomes
which is equivalent to
which is clearly satisfied.

##### 5.2. Global Results for System (4)

Lemma 17. *System (4) has no minimal period-two solution.*

*Proof. *The second iterate of the map is given as
Period-two solution satisfies which reduces to the following two equations:
Equation (79) is exactly the equilibrium equation (65). Using (65) in (78) we obtain after some elementary simplification that period-two solution satisfies (66). This shows that System (4) has no minimal period-two solution.

Lemma 18. *The maps and associated with System (4) have the following properties.*(i)*If , then the maps and are injective.*(ii)*If , then for all , .**Consequently, satisfies condition, when .*

*Proof. *(i) We will prove that is injective which will imply the injectivity of . The condition
is reduced to the following two conditions:
which implies that
and , when . This implies and so .

(ii) A direct calculation shows that
which proves our statement.

The statement on condition follows from Theorem 5.

Theorem 19. *Consider System (4). Assume that . Then there exists a set which is invariant subset of the basin of attraction of . The set is a graph of a strictly increasing continuous function of the first variable on an interval and separates into two connected and invariant components, namely,
**
which satisfy the following.**(i) If , then
**(ii) If ,
*

*Proof. *Assume that . Then by Lemma 18 and Theorem 4, every solution of System (4) has eventually monotone components and , which shows that every bounded solution converges to a period-two solution. In view of Lemma 17, there are no minimal period-two solutions and so every bounded solution of System (4) converges to the equilibrium . In view of Theorems 7 and 8, there is a set with described properties. Consequently, the regions and are invariant for and every solution in () is asymptotic to the unstable manifold and so the statement of the theorem follows.

Theorem 20. *Consider System (4). Assume that . Then System (4) can be decoupled and written as
**
Every solution of System (87) has eventually monotone subsequences and . Every bounded solution converges to the unique positive equilibrium. Every unbounded solution approaches either or .*

*Proof. *Using the condition in the second equation of System (4) gives
and so which shows that System (4) has an invariant of the form:
Using (89), System (4) is reduced to System (87). Both equations of System (87) are first order difference equations with decreasing functions and so by Theorem 1.19 [7], the subsequences of even and odd indexes are eventually monotonic and so every bounded solution converges to a period-two solution. An immediate checking shows that neither one of the two equations of System (87) has period-two solutions. For example, the unique equilibrium of the first equation of System (87) satisfies
while period-two solution satisfies which becomes and so is reduced to (90). Thus every bounded solution converges to the unique equilibrium.

The result for unbounded solutions follows immediately from (89).

#### Acknowledgment

The authors are grateful to one of the anonymous referees for pointing out the equation of the stable manifold for (2) and for observing the alternative method of the proof of Theorem 1 mentioned in Remark 11.