#### Abstract

A rigorous mathematical characterization for early-stage spatial and temporal patterns formation in a Leslie-Gower predator-prey model with cross diffusion is investigated. Given any general perturbation near an unstable constant equilibrium, we prove that its nonlinear evolution is dominated by the corresponding linear dynamics along a fixed finite number of the fastest growing modes.

#### 1. Introduction

Since Turing proposed the striking idea of “diffusion-driven instability” in 1952 [1], reaction-diffusion systems are often employed to investigate chemical and biological pattern formations and have received much attention from the scientists [2–7]. However, most of the works concentrate on pattern formation in the case of linear instability, and there is a little discussion about the nonlinear effect of a reaction-diffusion system on the evolution of a nonuniform pattern.

In general, nonlinear instability is treated with great delicacy and difficulty. At first, nonlinear instability was established for nondissipative systems [8–11]. In 2004, Guo et al. [12] established nonlinear instability for an unstable Kirchhoff ellipse. Based upon a precise linear analysis, they found that the dynamics of general perturbation can be characterized by the linear dynamics of the fastest growing modes. This marks a beginning of a quantitative description of instability. Subsequently, Guo and Hwang dealt with nonlinear stability for a Keller-Segel model in [13] and described the early-stage pattern formation in that model.

Recently, Guo and Hwang considered the following reaction-diffusion system [14] in a box with the homogeneous Neumann boundary conditions. In system (1), , denote the densities of two interactive species at time , the functions , are their diffusion rates, and , are the reaction functions. The classical Turing instability and Turing patterns were studied under some suitable conditions. Their result shows that the nonlinear evolution of patterns is dominated by the corresponding linear dynamics along a fixed finite number of the fastest growing modes over a time period.

In this paper, we consider the following Leslie-Gower predator-prey model with cross diffusion: where and represent the densities of the species prey and predator, respectively. The parameters , , , , , , and are all positive constants, where and are the intrinsic growth rates of the prey and predator, is the predation rate, and the term is a modified Leslie-Gower term [15]. The constants , , called diffusion coefficients, represent the natural tendency of each species to diffuse to areas of smaller population concentration, and , called a cross-diffusion coefficient, expresses the population flux of the predator resulting from the presence of the prey species. For more ecological backgrounds about this model, one can refer to [15–17].

System (2) and its variants were studied widely for pattern formation by applying the bifurcation theory and the degree theory [6, 18–20] in the case of linear instability. Inspired by the works [13, 14], in this paper, we attempt to study the nonlinear instability for this system and give a rigorous mathematical characterization for the nonlinear evolution of pattern by using a bootstrap technique. The mathematical approach in this paper is similar in spirit to that of [13, 14]. However, our problem (2) is much more complex. Notice that the diffusion term of the predator equation in the model (2) is In some sense, the coupled degree in (2) is stronger than that in (1). As a result, our analysis here, especially in establishing estimates for nonlinear terms and , is much more difficult and requires some techniques beyond those of [13, 14].

It is obvious that (2) has a unique positive equilibrium if and only if , where Let , be the perturbation around and still denote it by . Then, the perturbation satisfies the following strongly coupled equations: where

This paper is organized as follows. In Section 2, the growing modes in the linearized system are studied, which are important for our later discussions. Section 3 gives some estimates for the perturbation. The key is to control the nonlinear growth of high-order energy. In Section 4, the nonlinear instability is obtained.

#### 2. Growing Modes in the Linearized System

The corresponding linearized system of (5) takes the form of We use to denote a column vector and let , . Then, are eigenvalues of on under the homogeneous Neumann boundary condition, and the corresponding normalized eigenfunctions are given by This set of eigenfunctions forms an orthonormal basis in .

We look for a normal mode to be the linear system (7) of the following form: where is a complex number and is a vector; they depend on . Substituting (9) into (7), we have System (7) possesses a nontrivial normal mode if and only if which is equivalent to Thus, we deduce the following well-known aggregation (i.e., linear instability) criterion by requiring that there exists a , such that the constant term in (12) is In this paper, we always assume that there exists a , such that (13) holds. Then, the discriminant of (12) is and the coefficient of is positive, since (13) implies

For given , we denote the corresponding eigenvalues by and eigenvectors by . We split it into three cases for the linear analysis.(1). Let , and let be two distinct real roots with , being the corresponding (linearly independent) real eigenvectors. It is easy to see that Denote Clearly, for . Note that there are only finitely many in and . Therefore, there are only finitely many linear growing modes, such that the constant equilibrium of (5) is unstable. Furthermore, we define Then, .(2). Let . In this case, (12) possesses repeated real eigenvalues. Consider The corresponding eigenvectors are and we can find another independent vector , satisfying (3). The complex case is where (12) possesses a pair of complex eigenvalues with a negative real part. Denote , and for any , denote Then, where and are linearly independent vectors.

Given any initial perturbation , we can expand it as follows: where , , , , , and are constants, and The unique solution to (7) is given by

In the sequel, the constant will only depend on the domain and the dimension , and the generic constants , , , , and so forth will depend on , , and the parameters , , , , , , and . Our main result of this section is the following lemma.

Lemma 1. *Assume that the instability criterion (13) is valid. Suppose that
**
is a solution to the linearized system (7) with the initial condition . Then, there exists a constant , such that
**
for all . *

*Proof. *We first consider the case for . For any ,
where is given by (14). Applying Cramer’s rule to (25), we have
where
It follows from (14) that there exist positive constants and
such that for all . Hence, for any ,
and by (12),
Thus,
Consequently, there exists a positive constant , such that
for any . Substituting this into (33) yields
for any , where . We thus obtain
Since
there exists a constant , such that
for any .

For any and , as is an increasing function of , we denote
With the help of (30) and (31), we get
where only depends on , , , , , and . Hence, we conclude that, for any , there exists a positive constant , such that

For all and , by some similar arguments as above we can show that there exist positive constants and , such that

Next, we derive the energy estimate in for . Recall that is an orthonormal basis in . Then,
where
From (43) and (44), we obtain
Thus,
where . Finally, for any , we have

For finite time , we multiply the first and second equations of (7) by and , respectively, then add them and use the integration by parts to get
Firstly, we claim that the integrand of the second integral in (50) satisfies
for some positive constant . Obviously, it suffices to require that
This is equivalent to
Denote

On the other hand, the term on the right of (50) is
Taking , and substituting (51) and (55) into (50), we get
Integrating (56) from to leads to
If , then it follows from (57) that
thus, the Gronwall inequality implies
Consequently, there exists a positive constant , such that
for all due to the boundedness of

If , in the same way as above, there exists a positive constant , such that
The proof is completed by taking .

#### 3. The Estimates for the Solutions of the Full System (5)

The general theory in [21] guarantees that (5) has a unique nonnegative local solution. The results can be summarized as follows.

Lemma 2. *Suppose that is a solution of the full system (5). For and , there exist a and a constant , such that
**
if . *

Denote In order to derive the estimate for the solution of (5), we first prove the following energy estimates.

Lemma 3. *Suppose that is a solution of the full system (5). Then,
**
for . *

*Proof. *We first notice that system (5) preserves the evenness of the solution; that is, if is a solution to (5), then , , and are also solutions of (5). We can regard system (5) as a special case with the evenness of the periodic problem by a reflective and an even extension. For this reason, we may assume periodicity at the boundary of the extended . Taking the second order partial derivative of the first equation of (5), multiplying , and integrating over the domain to get
where
are the linear and nonlinear terms, respectively, then, we have
Similarly, taking the second order partial derivative of the second equation of (5), multiplying , and integrating over the domain to get
where
thus,
Substituting (70) (66) into (69) and (73) to get
where
we apply the Sobolev imbedding to control the norm by
for . From the Hölder inequality, the Poincaré inequality, and the Sobolev imbedding theorem, it follows that
for . Recall the even extension of (5), and the solution satisfies
By (76) and (77), we find that
where is a universal constant. Therefore, when , it follows from (76) and (79) that
Applying the Young inequality to get
which is combined with the interpolation inequality and the -Young inequality to imply
where is a positive constant, in the same way as above, we obtain that the second integral satisfies
Substituting (80)–(83) into (74), take to get

Similar as the proof of Lemma 1, we proceed in the two cases: and . Then, we conclude
So, the even extension implies

Next, we control the growth of in terms of its growth.

Lemma 4. *Suppose that is a solution of the full system (5), such that
**
Then,
**
where . *

*Proof. *We first consider the second-order derivatives of . From Lemmas 3 and 4 and our assumption for , we have
By an integration from 0 to , we deduce that
For the first derivations of , we apply the Poincaré inequality to get
Applying the even extension, we have
It follows from (88)–(93) that
where .

#### 4. Nonlinear Instability and Pattern Formation

Let be a fixed constant. For an arbitrary positive number , we define the escape time by or equivalently