Instability Induced by Cross-Diffusion in a Predator-Prey Model with Sex Structure

Abstract

In this paper, we consider a cross-diffusion predator-prey model with sex structure. We prove that cross-diffusion can destabilize a uniform positive equilibrium which is stable for the ODE system and for the weakly coupled reaction-diffusion system. As a result, we find that stationary patterns arise solely from the effect of cross-diffusion.

1. Introduction

Sex ratio means the comparison between the number of male and female in the species. The sex ratio is generally regarded as 1 : 1. But for wildlife, the sex ratio of species varies with the category, environment condition, community behavior, orientation and heredity, and so forth. The animal’s sex ratio in the different life history stages may vary with different animals. Take a bird as an example: the number of the older males is larger than that of the older females with the increase in age, which is contrary to the case of the mammal where the number of the older females is larger than that of the older males with the increase in age [1, 2]. Sex ratio is the basis of analyzing the dynamic state of different species, the variation of which has a huge influence on the dynamic state of the species [1–7]. Abrahams and Dill [5] have provided evidence that male and female guppies forage differently in the presence of predators and that sexual differences in the energetic equivalence of the risk of predation exist. Eubanks and Miller [6] have found that female Gladicosa pulchra (Lycosidae) wolf spiders climb trees significantly more often than males in the presence of forest floor predators. It is found that the sex of voles affects the risk of predation by mammals and female voles are more easily predated than male voles in [7].

Incorporating the sex of prey in a classical Lotka-Volterra model, Liu et al. [1] considered the following sex-structure model:𝑢^′(𝑡)=𝑏₁𝑣−𝑢𝐷₁+𝑘𝑢+𝑘𝑣+𝑐₁𝑤,𝑣^′(𝑡)=𝑣𝑏₂−𝐷₂−𝑘𝑢−𝑘𝑣−𝑐₁𝑤,𝑤^′(𝑡)=𝑤−𝐷₃+c₂𝑢+𝑐₂𝑣−𝑐₃𝑤, (1.1) where 𝑢, 𝑣, and 𝑤 are the population densities of the male prey, the female prey, and the predator species respectively. The parameters 𝐷₁, 𝐷₂, and 𝐷₃ are their mortality rates, 𝑏₁ and 𝑏₂ are the birth rates of the male prey and the female prey, 𝑐₁ and 𝑐₂ are the predation rate and the conversion rate of the predators, and 𝑘 and 𝑐₃ are the intraspecific competition rates of the prey and predators. All the parameters in model (1.1) are positive.

If 𝛽=𝑏₂−𝐷₂>0, then two obvious nonnegative equilibria of model (1.1) are 𝐮₀=(0,0,0) and 𝐮₁=(𝑢₁,𝑣₁,0), where𝑢₁=𝑏₁𝛽𝑘𝑏₁+𝐷₁+𝛽,𝑣₁=𝛽𝐷₁+𝛽𝑘𝑏₁+𝐷₁+𝛽. (1.2) Moreover, model (1.1) has a positive equilibrium if and only if𝑅∶=𝑐₂𝛽−𝑘𝐷₃>0. (H1) In this case the positive equilibrium is uniquely given by 𝐮=(𝑢,𝑣,𝑤), where𝑏𝑢=¹𝑐₃𝛽+𝑐₁𝐷₃𝑏₁+𝐷₁+𝛽𝑐₃𝑘+𝑐₁𝑐₂𝐷,𝑣=¹+𝛽𝑐₃𝛽+𝑐₁𝐷₃𝑏₁+𝐷₁+𝛽𝑐₃𝑘+𝑐₁𝑐₂,𝑐𝑤=²𝛽−𝑘𝐷₃𝑐₃𝑘+𝑐₁𝑐₂. (1.3) It turns out that 𝑅 plays an important role in determining the stability of 𝐮₁ and 𝐮 [1]. To be precise, 𝐮₁ is locally asymptotically stable if 𝑅<0, while 𝐮 is locally asymptotically stable if 𝑅>0. This shows that a uniform coexistence state exists and is stable when the intrinsic growth rate 𝛽 of the female prey is larger than the critical value 𝑘𝐷₃/𝑐₂.

Taking account of the inhomogeneous distribution of the prey and the predator in different spatial locations within a fixed bounded domain Ω at any given time, and the natural tendency of each species to diffuse to areas of smaller population concentration, Liu and Zhou in [8] investigated the following weakly coupled reaction-diffusion system:𝑢_𝑡−𝑑₁Δ𝑢=𝑏₁𝑣−𝑢𝐷₁+𝑘𝑢+𝑘𝑣+𝑐₁𝑤𝑣,𝑥∈Ω,𝑡>0,_𝑡−𝑑₂Δ𝑣=𝑣𝛽−𝑘𝑢−𝑘𝑣−𝑐₁𝑤𝑤,𝑥∈Ω,𝑡>0,_𝑡−𝑑₃Δ𝑤=𝑤−𝐷₃+𝑐₂𝑢+𝑐₂𝑣−𝑐₃𝑤𝜕,𝑥∈Ω,𝑡>0,_𝜂𝑢(𝑥,𝑡)=𝜕_𝜂𝑣(𝑥,𝑡)=𝜕_𝜂𝑤()𝑢()𝑥,𝑡=0,𝑥∈𝜕Ω,𝑡>0,𝑥,0=𝑢₀(𝑥),𝑣(𝑥,0)=𝑣₀(𝑥),𝑤(𝑥,0)=𝑤₀(𝑥),𝑥∈Ω, (1.4) where 𝜂 is the outward unit normal vector of the boundary 𝜕Ω which is smooth, 𝜕_𝜂=𝜕/𝜕_𝜂. The homogeneous Neumann boundary condition indicates that the predator-prey system is self-contained with zero population flux across the boundary. The constants 𝑑₁, 𝑑₂, and 𝑑₃, called diffusion coefficients, are positive, and the initial values 𝑢₀(𝑥), 𝑣₀(𝑥), and 𝑤₀(𝑥) are nonnegative smooth functions which are not identically zero. Liu and Zhou in [8] found that the nonnegative constant steady states have the same stability properties as the ODE model (1.1). Therefore, Turing instability cannot occur for this reaction-diffusion system.

However, in model (1.4), only diffusion of each individual species is taken into account. In some cases, the reality is that the female prey is easily predated because of physiological factor, while the male prey can congregate and form a huge group to protect itself from the attack of the predators [7, 9]; therefore, the predators tend to keep away from their male prey. Similarly as in [10–12], we model this by the cross-diffusion term Δ(𝑑₃𝑤+𝑑₄𝑢𝑤) for the predators, where 𝑑₄>0, called the cross-diffusion coefficient. Thus, the cross-diffusion system that we will study is the following:𝑢_𝑡−𝑑₁Δ𝑢=𝑏₁𝑣−𝑢𝐷₁+𝑘𝑢+𝑘𝑣+𝑐₁𝑤∶=𝐺₁()𝑣𝑢,𝑣,𝑤,𝑥∈Ω,𝑡>0,_𝑡−𝑑₂Δ𝑣=𝑣𝛽−𝑘𝑢−𝑘𝑣−𝑐₁𝑤∶=𝐺₂()𝑤𝑢,𝑣,𝑤,𝑥∈Ω,𝑡>0,_𝑡−Δ𝑑₃𝑤+𝑑₄𝑢𝑤=𝑤−𝐷₃+𝑐₂𝑢+𝑐₂𝑣−𝑐₃𝑤∶=𝐺₃()𝜕𝑢,𝑣,𝑤,𝑥∈Ω,𝑡>0,_𝜂𝑢(𝑥,𝑡)=𝜕_𝜂𝑣(𝑥,𝑡)=𝜕_𝜂𝑤()𝑢()𝑥,𝑡=0,𝑥∈𝜕Ω,𝑡>0,𝑥,0=𝑢₀(𝑥),𝑣(𝑥,0)=𝑣₀(𝑥),𝑤(𝑥,0)=𝑤₀(𝑥),𝑥∈Ω. (1.5) To our knowledge, only a few works investigated the effect of cross-diffusion on population structure and dynamics in the above model. Recently, H. Xu and S. Xu in [13] investigated the global existence of solutions for the corresponding full SKT model of (1.5) when the space dimension is less than ten.

An interesting feature of (1.5) is that the interaction between the predators and the male prey gives rise to a cross-diffusion term. The resulting mathematical model is a strongly coupled system of three equations which is mathematically much more complex than those considered earlier. In this paper, we will show that cross-diffusion can destabilize the uniform equilibrium 𝐮 which is stable for models (1.1) and (1.4). Moreover, we will demonstrate that the nonlinear dispersive force can give rise to a spatial segregation of these species.

Our paper is organized as follows. In Section 2, we analyze the local stability of 𝐮 for (1.5) and calculate the fixed point index, which is important for our later discussions on the existence of nonconstant positive steady states. In Section 3, we prove global asymptotic stability of 𝐮 with 𝑑₄=0, that is, when no cross-diffusion occurs in the model. This implies that cross-diffusion has a destabilizing effect. In Section 4, we establish a priori upper and lower bounds for all possible positive steady states of (1.5). In Section 5, we study the global existence of nonconstant positive steady states of (1.5) for suitable values of the parameters. This is done by using the Leray-Schauder degree theory and the results obtained in Sections 2, 3, and 4. In Section 6, we discuss the nonexistence of nonconstant positive steady states of (1.5). In the last section, we give a brief discussion about our model.

2. Local Stability Analysis and Fixed Point Index of 𝑢

Let 𝐮=(𝑢,𝑣,𝑤)^𝑇, Φ(𝐮)=(𝑑₁𝑢,𝑑₂𝑣,𝑑₃𝑤+𝑑₄𝑢𝑤)^𝑇, and 𝐆(𝐮)=(𝐺₁(𝐮),𝐺₂(𝐮),𝐺₃(𝐮))^𝑇. Then the stationary problem of (1.5) can be written as−ΔΦ(𝐮)=𝐆(𝐮)inΩ;𝜕_𝜂𝐮=0on𝜕Ω. (2.1) In this section, we study the linearization of (2.1) at 𝐮 and calculate the fixed point index.

Similar to [14, 15], let 0=𝜇₁<𝜇₂<𝜇₃<𝜇₄⋯ be the eigenvalues of the operator −Δ on Ω with the homogeneous Neumann boundary condition, and let 𝐸(𝜇_𝑖) be the eigenspace corresponding to 𝜇_𝑖 in 𝐻¹(Ω). Let {𝜙_𝑖𝑗∶𝑗=1,2,…,dim𝐸(𝜇_𝑖)} be the orthonormal basis of 𝐸(𝜇_𝑖), 𝐗=[𝐻¹(Ω)]³, and 𝐗_𝑖𝑗={𝐜𝜙_𝑖𝑗∶𝐜∈ℝ³}. Then𝐗=^+∞_𝑖=1𝐗_𝑖,𝐗_𝑖=^{dim𝐸(𝜇𝑖)}_𝑗=1𝐗_𝑖𝑗. (2.2)

Let 𝐘=[𝐶¹(Ω)]³, 𝐘⁺={𝐮∈𝐘∶𝑢,𝑣,𝑤>0 on Ω}, and 𝐵(𝐶)={𝐮∈𝐘∶𝐶⁻¹<𝑢,𝑣,𝑤<𝐶 on Ω} for 𝐶>0. Since detΦ_𝐮(𝐮)=𝑑₁𝑑₂(𝑑₃+𝑑₄𝑢)>0 for all nonnegative 𝐮, Φ_𝐮^−𝟏(𝐮) exists and det{Φ_𝐮^−𝟏(𝐮)} is positive. Hence, 𝐮 is a positive solution to (2.1) if and only if𝐅(𝐮)∶=𝐮−(𝐈−Δ)⁻¹Φ_𝐮⁻¹(𝐮)𝐆(𝐮)+∇𝐮Φ_𝐮𝐮(𝐮)∇𝐮+𝐮=0in𝐘⁺, (2.3)

where (𝐈−Δ)⁻¹ is the inverse of 𝐈−Δ under homogeneous Neumann boundary conditions.

Further, we note that 𝐷_𝐮𝐅(𝐮)=𝐈−(𝐈−Δ)⁻¹{Φ_𝐮⁻¹(𝐮)𝐆_𝐮(𝐮)+𝐈} and 𝜆 is an eigenvalue of 𝐷_𝐮𝐅(𝐮) if and only if, for some 𝑖≥1, it is an eigenvalue of the matrix𝐁_𝑖1∶=𝐈−1+𝜇_𝑖Φ_𝐮⁻¹(𝐮)𝐆_𝐮(𝐮)=1+𝐈1+𝜇_𝑖𝜇_𝑖𝐈−Φ_𝐮⁻¹(𝐮)𝐆_𝐮(𝐮). (2.4)

Writing𝐻𝜇=𝐻𝐮;𝜇∶=det𝜇𝐈−Φ_𝐮⁻¹(𝐮)𝐆_𝐮(𝐮), (2.5) we see that if 𝐻(𝜇_𝑖)≠0, then for each integer 1≤𝑗≤dim𝐸(𝜇_𝑖), the number of negative eigenvalues of 𝐷_𝐮𝐅(𝐮) on 𝐗_𝑖𝑗 is odd if and only if 𝐻(𝜇_𝑖)<0. As a consequence, we have the following proposition.

Proposition 2.1 (see [16]). Suppose that, for all 𝑖≥1, 𝐻(𝜇_𝑖)≠0. Then index(𝐅(⋅),𝐮)=(−1)^𝛾, (2.6) where 𝛾=_{𝜇𝑖≥1,𝐻𝑖<0}dim𝐸𝜇_𝑖. (2.7)

To facilitate our computation of index(𝐹(⋅),𝐮), we need to determine the sign of 𝐻(𝜇_𝑖). In particular, as the aim of this paper is to study the existence of stationary patterns of (2.1) with respect to the cross-diffusion coefficient 𝑑₄, we will concentrate on the dependence of 𝐻(𝜇_𝑖) on 𝑑₄. At this point, we note that 𝐻(𝜇)=det{Φ_𝐮⁻¹(𝐮)}det{𝜇Φ_𝐮(𝐮)−𝐆_𝐮(𝐮)}. Since det{Φ_𝐮⁻¹(𝐮)} is positive, we will need only to consider det{𝜇Φ_𝐮(𝐮)−𝐆_𝐮(𝐮)}. By𝐆_𝐮(𝐮)=−𝑘𝑢−𝐷₁−𝛽𝑏₁−𝑘𝑢−𝑐₁𝑢−𝑘𝑣−𝑘𝑣−𝑐₁𝑐₂𝑣𝑤𝑐₂𝑤−𝑐₃𝑤,𝚽_𝐮(𝐮)=𝑑₁000𝑑₂0𝑑₄𝑤0𝑑₃+𝑑₄𝑢, (2.8) we havedet𝜇Φ_𝐮(𝐮)−𝐆_𝐮(𝐮)=𝐶₃𝑑₄𝜇³+𝐶₂𝑑₄𝜇²+𝐶₁𝑑₄𝜇−det𝐆_𝐮(𝐮)∶=𝒞𝑑₄;𝜇, (2.9) where𝐶₃𝑑₄=𝑑₁𝑑₂𝑑₃+𝑑₄,𝐶₂𝑢𝑑₄=𝑑₁𝑑₂𝑐₃𝑤+𝑑₁𝑘𝑑₃+𝑑₄𝑢𝑣+𝑑₂𝑑₃+𝑑₄𝑢𝑘𝑢+𝐷₁+𝛽−𝑑₂𝑑₄𝑐₁𝐶𝑢𝑤,₁𝑑₄=𝑑₁𝑘𝑐₃𝑣𝑤+𝑑₂𝑐₃𝑘𝑢+𝐷₁+𝛽𝑤+𝑑₃+𝑑₄𝑢𝑘𝑢+𝐷₁+𝛽𝑘𝑣+𝑑₄𝑐₁𝑣𝑤𝑘𝑢−𝑏₁−𝑐₁𝑢−𝑑₂𝑐₂𝑤+𝑘𝑑₄𝑣𝑤+𝑐₁𝑐₂𝑑₁𝑣𝑤−𝑑₃+𝑑₄𝑢𝑘𝑣𝑘𝑢−𝑏₁,det𝐆_𝐮(𝐮)=𝑣𝑤−𝑘𝑐₃+𝑐₁𝑐₂𝑘𝑢+𝐷₁+𝛽+𝑐₁𝑐₂+𝑘𝑐₃𝑘𝑢−𝑏₁. (2.10)

Notice that 𝑘𝑢−𝑏₁<0; thus det𝐆_𝐮(𝐮)<0. We consider the dependence of 𝒞 on 𝑑₄. Let 𝜇₁(𝑑₄), 𝜇₂(𝑑₄), and 𝜇₃(𝑑₄) be the three roots of 𝒞(𝑑₄;𝜇)=0 with Re{𝜇₁(𝑑₄)}≤Re{𝜇₂(𝑑₄)}≤Re{𝜇₃(𝑑₄)}. It follows that 𝜇₁(𝑑₄)𝜇₂(𝑑₄)𝜇₃(𝑑₄)<0 from det𝐆_𝐮(𝐮)<0. Thus, among 𝜇₁(𝑑₄), 𝜇₂(𝑑₄), 𝜇₃(𝑑₄) at least one is real and negative, and the product of the other two is positive.

Consider the following limits:lim_𝑑4→∞𝐶₃𝑑₄𝑑₄=𝑑₁𝑑₂𝑢∶=𝑎₃,lim_𝑑4→∞𝐶₂𝑑₄𝑑₄=𝑑₁𝑘𝑢𝑣+𝑑₂𝑏𝑢𝑘𝑢+¹𝑣𝑢−𝑑₂𝑐₁𝑢𝑤=𝑑₁𝑘𝑢𝑣+𝑑₂𝑘𝑢²+𝑏₁𝑣−𝑐₁𝑢𝑤∶=𝑎₂,lim_𝑑4→∞𝐶₁𝑑₄𝑑₄𝑏=𝑢𝑘𝑢+¹𝑣𝑢𝑘𝑣+𝑐₁𝑣𝑤𝑘𝑢−𝑏₁−𝑐₁𝑢𝑘𝑣𝑤−𝑢𝑘𝑣𝑘𝑢−𝑏₁=𝑏₁𝑣𝛽−2𝑐₁𝑤∶=𝑎₁. (2.11)

Therefore, 𝑎₁<0 if𝛽<2𝑐₁𝑤. (H2) In the following, we restrict our attention to 𝛽<2𝑐₁𝑤. In this range, 𝑎₁<0 and 𝐶₁(𝑑₄)<0 for all sufficiently large 𝑑₄. Notice thatlim_𝑑4→∞𝒞𝑑₄;𝜇𝑑₄=𝑎₃𝜇³+𝑎₂𝜇²+𝑎₁𝜇=𝜇𝑎₃𝜇²+𝑎₂𝜇+𝑎₁ (2.12)

and 𝑎₁<0<𝑎₃. A continuity argument shows that, when 𝑑₄ is large, 𝜇₁(𝑑₄) is real and negative. Furthermore, as 𝜇₂(𝑑₄)𝜇₃(𝑑₄)>0, 𝜇₂(𝑑₄) and 𝜇₃(𝑑₄) are real and positive, andlim_𝑑4→∞𝜇₁𝑑₄=−𝑎₂−𝑎²₂−4𝑎₁𝑎₃2𝑎₃<0,lim_𝑑4→∞𝜇₂𝑑₄=0,lim_𝑑4→∞𝜇₃𝑑₄=−a₂+𝑎²₂−4𝑎₁𝑎₃2𝑎₃∶=𝜇>0. (2.13) Thus we have the following proposition.

Proposition 2.2. Assume that (H1) and (H2) hold. Then there exists a positive number 𝑑^∗₄ such that, when 𝑑₄≥𝑑^∗₄, the three roots 𝜇₁(𝑑₄), 𝜇₂(𝑑₄), 𝜇₃(𝑑₄) of 𝒞(𝑑₄;𝜇)=0 are all real and satisfy (2.13). Moreover, for all 𝑑₄≥𝑑^∗₄, −∞<𝜇₁𝑑₄<0<𝜇₂𝑑₄<𝜇₃𝑑₄,𝒞𝑑₄;𝜇<0,𝜇∈−∞,𝜇₁𝑑₄∪𝜇₂𝑑₄,𝜇₃𝑑₄,𝒞𝑑₄;𝜇>0,𝜇∈𝜇₁𝑑₄,𝜇₂𝑑₄∪𝜇₃𝑑₄,+∞. (2.14)

Remark 2.3. Proposition 2.2 gives a criterion for the instability of 𝐮 when 𝜇>𝜇₂ and the cross-diffusion coefficient 𝑑₄ is large enough. We further check conditions (H1) and (H2). Let the parameters 𝑑₁, 𝑑₂, 𝑑₃, 𝑏₁, 𝐷₁, 𝑘, 𝑐₁, 𝑐₂, and 𝑐₃ be fixed. Condition (H1) is equivalent to 𝛽>𝑘𝐷₃/𝑐₂, and condition (H2) is equivalent to 𝛽>2𝑘𝑐₁𝐷₃/𝛾₁ for some 𝛾₁∶=𝑐₁𝑐₂−𝑘𝑐₃>0. Notice that 2𝑐₁/𝛾₁>1/𝑐₂, so there exists an unbounded region 𝑈₁={(𝐷₃,𝛽)∈ℝ²₊∶𝛽>2𝑘𝑐₁𝐷₃/𝛾₁}, such that for any (𝐷₃,𝛽)∈𝑈₁, 𝐮 is an unstable equilibrium with respect to (1.5) when 𝜇>𝜇₂ and the cross-diffusion coefficient 𝑑₄ is sufficiently large.

3. Global Asymptotic Stability of 𝑢 for (1.4)

The aim of this section is to prove Theorem 3.2 which shows that model (1.4) has no nonconstant positive steady state no matter what the diffusion coefficients 𝑑₁, 𝑑₂, and 𝑑₃ are; in other words, diffusion alone (without cross-diffusion) cannot drive instability and cannot generate patterns for this predator-prey model. For this, we will make use of the following result.

Lemma 3.1 (see [17]). Let 𝑎 and 𝑏 be positive constants. Assume that 𝜑,𝜓∈𝐶¹([𝑎,+∞)), 𝜓(𝑡)≥0, and 𝜑 is bounded from below. If 𝜑′(𝑡)≤−𝑏𝜓(𝑡) and 𝜓′(𝑡) is bounded in [𝑎,+∞), then lim_𝑡→∞𝜓(𝑡)=0.

Theorem 3.2. Let the parameters 𝑑₁, 𝑑₂, 𝑑₃, 𝑏₁, 𝐷₁, 𝐷₃, 𝑘, 𝑐₁, 𝑐₂, 𝑐₃, and 𝛽 be fixed positive constants that satisfy (H1) and 𝑏²₁<4𝑘𝑢𝐷₁+𝑏₁. (H3) Let (𝑢,𝑣,𝑤) be a positive solution of (1.4). Then ‖𝑢(⋅,𝑡)−𝑢‖_𝐿2(Ω)⟶0,‖𝑣(⋅,𝑡)−𝑣‖_𝐿2(Ω)‖𝑤()‖⟶0,⋅,𝑡−𝑤_𝐿2(Ω)⟶0as𝑡⟶∞. (3.1)

Proof. Notice from [8] that 𝐮 is uniformly and locally asymptotically stable in the sense of [18]. We only need to prove the global stability of 𝐮. Define 𝑉₁()=1𝑢,𝑣,𝑤2_Ω(𝑢−𝑢)²𝑑𝑥+𝜆_Ω𝑣𝑣−𝑣−𝑣ln𝑣𝑑𝑥+𝜌_Ω𝑤𝑤−𝑤−𝑤ln𝑤𝑑𝑥, (3.2)where 𝜆=𝑢, 𝜌=𝑐₁𝑢/𝑐₂. Obviously, 𝑉₁(𝑢,𝑣,𝑤) is nonnegative and 𝑉₁(𝑢,𝑣,𝑤)=0 if and only if (𝑢,𝑣,𝑤)=(𝑢,𝑣,𝑤). The time derivative of 𝑉₁(𝑢,𝑣,𝑤) for the system (1.4) satisfies 𝑑𝑉₁(𝑢,𝑣,𝑤)=𝑑𝑡_Ω(𝑢−𝑢)𝑢_𝑡+𝜆𝑣−𝑣𝑣𝑣_𝑡+𝜌𝑤−𝑤𝑤𝑤_𝑡𝑑𝑥∶=−𝐼₁(𝑡)−𝐼₂(𝑡), (3.3)where 𝐼₁(𝑡)=_Ω𝑑₁|∇𝑢|²𝑑+𝜆²𝑣𝑣²|∇𝑣|²𝑑+𝜌³𝑤𝑤²|∇𝑤|²𝐼𝑑𝑥,₂(𝑡)=_Ω𝐷₁+𝑘𝑢+𝑘𝑢+𝑘𝑣+𝑐₁𝑤(𝑢−𝑢)²+𝜆𝑘(𝑣−𝑣)²+𝜌𝑐₃(𝑤−𝑤)²+2k𝑢−𝑏₁(≥𝑢−𝑢)(𝑣−𝑣)𝑑𝑥,_Ω𝐷₁+𝑘𝑢(𝑢−𝑢)²+𝜆𝑘(𝑣−𝑣)²+𝜌𝑐₃(𝑤−𝑤)²+2𝑘𝑢−𝑏₁(𝑢−𝑢)(𝑣−𝑣)𝑑𝑥. (3.4)If the matrix 𝐷₁1+𝑘𝑢22𝑘𝑢−𝑏₁0122𝑘𝑢−𝑏₁𝜆𝑘000𝜌𝑐₃ (3.5)is positive definite, then the quadratic form 𝐷₁+𝑘𝑢(𝑢−𝑢)²+𝜆𝑘(𝑣−𝑣)²+𝜌𝑐₃(𝑤−𝑤)²+2𝑘𝑢−𝑏₁(𝑢−𝑢)(𝑣−𝑣) (3.6)is positive definite. A direct calculation shows that the matrix is positive definite if (H3) holds. Meanwhile, for every 𝛿 such that 0<𝛿<min{𝑐₃𝜌,(4𝑘𝑢(𝐷₁+𝑏₁)−𝑏²₁)/(4(𝐷₁+2𝑘𝑢))}, we have 𝐼₂(𝑡)≥𝛿_Ω(𝑢−𝑢)²+(𝑣−𝑣)²+(𝑤−𝑤)²𝑑𝑥. (3.7)Thus, 𝑑𝑉₁(𝑢,𝑣,𝑤)𝑑𝑡≤−𝛿_Ω(𝑢−𝑢)²+(𝑣−𝑣)²+(𝑤−𝑤)²𝑑𝑥. (3.8)Similarly to [19, Theorem 2.1], we can prove that the solution (𝑢,𝑣,𝑤) is bounded, and so are the derivatives of ∫_Ω[(𝑢−𝑢)²+(𝑣−𝑣)²+(𝑤−𝑤)²]𝑑𝑥 by the equations in (1.4). Using Lemma 3.1, we have ‖𝑢(⋅,𝑡)−𝑢‖_𝐿2(Ω)⟶0,‖𝑣(⋅,𝑡)−𝑣‖_𝐿2(Ω)‖𝑤()‖⟶0,⋅,𝑡−𝑤_𝐿2(Ω)⟶0as𝑡⟶∞. (3.9)By the fact that 𝑉₁(𝑢,𝑣,𝑤) is decreasing for 𝑡≥0, it is obvious that (𝑢,𝑣,𝑤) is globally asymptotically stable, and the proof of Theorem 3.2 is completed.

Remark 3.3. Notice that condition (H3) is equivalent to 𝛾₂𝑏𝛽>¹𝑘𝑐₃+𝑐₁𝑐₂4𝑘−𝑐₁𝐷₃,𝛾₂=3𝑘𝑏₁𝑐₃+4𝑘𝑐₃𝐷₁−𝑏₁𝑐₁𝑐₂4𝑘𝑏₁+𝐷₁. (3.10)If 𝛾₂<0, it is easy to verify that −𝑐₁/𝛾₂>𝑘/𝑐₂. Hence, there exists an unbounded region 𝑈₂=𝐷₃,𝛽∈ℝ²₊∶𝛽>𝑘𝐷₃𝑐₂,𝛾₂𝑏𝛽>¹𝑘𝑐₃+𝑐₁𝑐₂4𝑘−𝑐₁𝐷₃, (3.11)such that for any (𝐷₃,𝛽)∈𝑈₂, 𝐮 is the unique positive steady state with respect to (1.4).

Remark 3.4. From Remarks 2.3 and 3.3, there exists an unbounded region 𝑈₃=𝑈₁∩𝑈₂=𝐷₃,𝛽∈ℝ²₊∶𝛾₁>0,𝛽>2𝑘𝑐₁𝛾₁𝐷₃,𝛾₂𝑏𝛽>¹𝑘𝑐₃+𝑐₁𝑐₂4𝑘−𝑐₁𝐷₃, (3.12)such that for any (𝐷₃,𝛽)∈𝑈₃, cross-diffusion can destabilize the uniform equilibrium 𝐮 of (1.5) when 𝜇>𝜇₂ and 𝑑₄ is sufficiently large.

4. A Priori Estimates

In the following, the generic constants 𝐶, 𝐶_∗, and so forth, will depend on the domain Ω and the dimension 𝑁. However, as Ω and the dimension 𝑁 are fixed, we will not mention the dependence explicitly. Also, for convenience, we will write Λ instead of the collective constants (𝑏₁,𝐷₁,𝐷₃,𝑐₁,𝑐₂,𝑐₃,𝑘,𝛽). The main purpose of this section is to give a priori positive upper and lower bounds for the positive solutions to (2.1) when 𝑅>0. For this, we will cite the following two results.

Lemma 4.1 (Harnack’s inequality [20]). Let 𝑤∈𝐶²(Ω)∩𝐶¹(Ω) be a positive solution to Δ𝑤(𝑥)+𝑐(𝑥)𝑤(𝑥)=0, where 𝑐∈𝐶(Ω), satisfying the homogeneous Neumann boundary condition. Then there exists a positive constant 𝐶_∗ which depends only on ‖𝑐‖_∞ such that max_Ω𝑤≤𝐶_∗min_Ω𝑤.

Lemma 4.2 (maximum principle [21]). Let 𝑔∈𝐶(Ω×ℝ¹) and 𝑏_𝑗∈𝐶(Ω), 𝑗=1,2,…,𝑁. (i)If 𝑤∈𝐶²(Ω)∩𝐶¹(Ω) satisfies Δ𝑤(𝑥)+^𝑁_𝑗=1𝑏_𝑗(𝑥)𝑤_𝑥𝑗+𝑔(𝑥,𝑤(𝑥))≥0inΩ,𝜕_𝜂𝑤≤0on𝜕Ω, (4.1) and 𝑤(𝑥₀)=max_Ω𝑤(𝑥), then 𝑔(𝑥₀,𝑤(𝑥₀))≥0.(ii) If 𝑤∈𝐶²(Ω)∩𝐶¹(Ω) satisfies Δ𝑤(𝑥)+^𝑁_𝑗=1𝑏_𝑗(𝑥)𝑤_𝑥𝑗+𝑔(𝑥,𝑤(𝑥))≤0inΩ,𝜕_𝜂𝑤≥0on𝜕Ω, (4.2) and 𝑤(𝑥₀)=min_Ω𝑤(𝑥), then 𝑔(𝑥₀,𝑤(𝑥₀))≤0.

Theorem 4.3 (upper bound). Let 𝑑 and 𝑑^∗ be two fixed positive constants. Assume that 𝑑_𝑖≥𝑑,𝑖=1,2,3, and 0≤𝑑₄≤𝑑^∗. Then every possible positive solution (𝑢,𝑣,𝑤) of (2.1) satisfies max_Ω𝑏𝑢≤¹𝑘,max_Ω𝛽𝑣≤𝑘,max_Ω𝑑𝑤≤1+^∗𝑏₁𝑐𝑑𝑘²𝑏₁+𝛽𝑐₃𝑘. (4.3)

Proof. A direct application of the maximum principle to (2.1) gives 𝑣≤𝛽/𝑘 on Ω. Let 𝑢(𝑥₀)=max_Ω𝑢. Using the maximum principle again, we have 𝑏₁𝑣(𝑥₀)≥𝑢(𝑥₀)[𝐷₁+𝑘𝑢(𝑥₀)+𝑘𝑣(𝑥₀)+𝑐₁𝑤(𝑥₀)]. Thus, 𝑘𝑢(𝑥₀)𝑣(𝑥₀)≤𝑏₁𝑣(𝑥₀) and 𝑢(𝑥₀)≤𝑏₁/𝑘.
Define 𝜑=𝑑₃𝑤+𝑑₄𝑢𝑤; then, 𝜑 satisfies−Δ𝜑=𝑤−𝐷₃+𝑐₂𝑢+𝑐₂𝑣−𝑐₃𝑤inΩ,𝜕_𝜂𝜑=0on𝜕Ω. (4.4)Let 𝜑(𝑥₁)=max_Ω𝜑. By Lemma 4.2, we have −𝐷₃+𝑐₂𝑢𝑥₁+𝑐₂𝑣𝑥₁−𝑐₃𝑤𝑥₁≥0. (4.5)It follows that 𝑤𝑥₁≤𝑐₂𝑐₃𝑢𝑥₁+𝑣𝑥₁≤𝑐₂𝑐₃𝑏₁𝑘+𝛽𝑘=𝑐₂𝑏₁+𝛽𝑐₃𝑘. (4.6)Hence, 𝜑𝑥₁=𝑑₃+𝑑₄𝑢𝑥₁𝑤𝑥₁≤𝑑₃+𝑑₄𝑏₁𝑘𝑐₂𝑏₁+𝛽𝑐₃𝑘,max_Ω𝑤=max_Ω𝜑𝑑₃+𝑑₄𝑢≤𝑑1+⁴𝑏₁𝑑₃𝑘𝑐₂𝑏₁+𝛽𝑐₃𝑘≤𝑑1+^∗𝑏₁𝑐𝑑𝑘²𝑏₁+𝛽𝑐₃𝑘 (4.7)for any 𝑑₃≥𝑑 and 0≤𝑑₄≤𝑑^∗.

Turning now to the lower bound, we first need some preliminary results.

Lemma 4.4. Let 𝑑_𝑖𝑗 be positive constants, 𝑖=1,2,3,4, 𝑗=1,2,…, and let (𝑢_𝑗,𝑣_𝑗,𝑤_𝑗) be the corresponding positive solution of (2.1) with 𝑑_𝑖=𝑑_𝑖𝑗. If (𝑢_𝑗,𝑣_𝑗,𝑤_𝑗)→(𝑢^∗,𝑣^∗,𝑤^∗) uniformly on Ω as 𝑗→∞ and (𝑢^∗,𝑣^∗,𝑤^∗) is a constant vector, then (𝑢^∗,𝑣^∗,𝑤^∗) must satisfy 𝑏₁𝑣^∗−𝑢^∗𝐷₁+𝑘𝑢^∗+𝑘𝑣^∗+𝑐₁𝑤^∗=0,𝛽−𝑘𝑢^∗−𝑘𝑣^∗−𝑐₁𝑤^∗=0,−𝐷₃+𝑐₂𝑢^∗+𝑐₂𝑣^∗−𝑐₃𝑤^∗=0. (4.8)Moreover, if 𝑢^∗, 𝑣^∗, and 𝑤^∗ are positive constants, then (𝑢^∗,𝑣^∗,𝑤^∗)=(𝑢,𝑣,𝑤).

Proof. It is easy to see that for all 𝑗, _Ω𝐺₁𝑢_𝑗,𝑣_𝑗,𝑤_𝑗𝑑𝑥=0. (4.9)If 𝐺₁(𝑢^∗,𝑣^∗,𝑤^∗)>0, then 𝐺₁(𝑢_𝑗,𝑣_𝑗,𝑤_𝑗)>0 when 𝑗 is large since (𝑢_𝑗,𝑣_𝑗,𝑤_𝑗)→(𝑢^∗,𝑣^∗,𝑤^∗). This is impossible. Similarly, 𝐺₁(𝑢^∗,𝑣^∗,𝑤^∗)<0 is impossible. Therefore, 𝐺₁(𝑢^∗,𝑣^∗,𝑤^∗)=0. The same argument shows that 𝛽−𝑘𝑢^∗−𝑘𝑣^∗−𝑐₁𝑤^∗=0 and −𝐷₃+𝑐₂𝑢^∗+𝑐₂𝑣^∗−𝑐₃𝑤^∗=0. Consequently, (𝑢^∗,𝑣^∗,𝑤^∗)=(𝑢,𝑣,𝑤).

Lemma 4.5. The system 𝑢_𝑡−𝑑₁Δ𝑢=𝑏₁𝑣−𝑢𝐷₁𝑣+𝑘𝑢+𝑘𝑣,𝑥∈Ω,𝑡>0,_𝑡−𝑑₂𝜕Δ𝑣=𝑣𝛽−𝑘𝑢−𝑘𝑣,𝑥∈Ω,𝑡>0,_𝜂𝑢=𝜕_𝜂𝑢()≥(≢)()≥(≢)𝑣=0,𝑥∈𝜕Ω,𝑡>0,𝑥,00,𝑣𝑥,00,𝑥∈Ω, (4.10) has a unique positive constant steady state (𝑢₁,𝑣₁) which is globally asymptotically stable, where 𝑢₁ and 𝑣₁ are given by (1.2).

Proof. Let 𝑉₂()=1𝑢,𝑣2_Ω𝑢−𝑢₁²𝑑𝑥+𝜌_Ω𝑣−𝑣₁−𝑣₁𝑣ln𝑣₁𝑑𝑥, (4.11)where 𝜌=(𝑏₁−𝑘𝑢₁)/𝑘=𝑏₁(𝑏₁+𝐷₁)/[𝑘(𝑏₁+𝐷₁+𝛽)]>0, and let (𝑢,𝑣) be a positive solution of (4.10). Then a direct computation gives 𝑑𝑉₂𝑑𝑡=−_Ω𝑑₁|∇𝑢|²+𝜌𝑑₂𝑣₁𝑣²|∇𝑣|²+𝐷₁+𝑘𝑢+𝑘𝑢₁+𝑘𝑣𝑢−𝑢₁²+𝜌𝑘𝑣−𝑣₁²𝑑𝑥≤0, (4.12)and 𝑑𝑉₂/𝑑𝑡=0 holds if and only if (𝑢,𝑣)=(𝑢₁,𝑣₁). By Lemma 3.1, we can conclude that (𝑢₁,𝑣₁) is globally asymptotically stable.

Theorem 4.6 (lower bound). Let 𝑑 and 𝑑^∗ be two fixed positive constants. Assume that 𝑑_𝑖≥𝑑, 𝑖=1,2,3, and 0≤𝑑₄≤𝑑^∗. Then there exists a positive constant 𝐶=𝐶(Λ,𝑑,𝑑^∗), such that every possible positive solution (𝑢,𝑣,𝑤) of (2.1) satisfies min_Ω𝑢,min_Ω𝑣,min_Ω𝑤≥𝐶. (4.13)

Proof. If the conclusion does not hold, then there exists a sequence {𝑑_1𝑗,𝑑_2𝑗,𝑑_3𝑗,𝑑_4𝑗}^∞_𝑗=1 with 𝑑_1𝑗,𝑑_2𝑗,𝑑_3𝑗≥𝑑 and 0≤𝑑_4𝑗≤𝑑^∗ such that the corresponding positive solution (𝑢_𝑗,𝑣_𝑗,𝑤_𝑗) of (2.1) satisfies minmin_Ω𝑢_𝑗,min_Ω𝑣_𝑗,min_Ω𝑤_𝑗⟶0,as𝑗⟶∞. (4.14)Moreover, we assume that 𝑑_𝑖𝑗→𝑑_𝑖∈[𝑑,∞] for 𝑖=1,2,3, and 𝑑_4𝑗→𝑑₄∈[0,𝑑^∗]. By Theorem 4.3 and the standard regularity theory for the elliptic equations, we may also assume that (𝑢_𝑗,𝑣_𝑗,𝑤_𝑗)→(𝑢,𝑣,𝑤) in [𝐶²(Ω)]³ for some nonnegative functions 𝑢,𝑣,𝑤. It is easy to see that (𝑢,𝑣,𝑤) also satisfies estimate (4.3), and min{min_Ω𝑢,min_Ω𝑣,min_Ω𝑤}=0. Moreover, we observe that, if 𝑑₁,𝑑₂,𝑑₃<∞, then (𝑢,𝑣,𝑤) satisfies (2.1).
Next we derive a contradiction for all possible cases.
Firstly, we consider the case 𝑑₁,𝑑₂,𝑑₃<∞.
(1)In view of (2.1), min_Ω𝑣=0 implies 𝑣=0 on Ω from the Harnack inequality. In this case, by the strong maximum principle and the Hopf boundary lemma, it follows that 𝑢=𝑤=0 on Ω. This is a contradiction to Lemma 4.4. Thus, min_Ω𝑣>0.(2) If min_Ω𝑢=0, we denote 𝑢(𝑥₀)=min_Ω𝑢=0. By the maximum principle we have 𝑏₁𝑣(𝑥₀)≤𝑢(𝑥₀)[𝐷₁+𝑘𝑢(𝑥₀)+𝑘𝑣(𝑥₀)+𝑐₁𝑤(𝑥₀)]=0, and so min_Ω𝑣=0. This is a contradiction to min_Ω𝑣>0. Thus min_Ω𝑢>0.(3) If min_Ω𝑤=0, let 𝜑=𝑑₃𝑤+𝑑₄𝑢𝑤. Then min_Ω𝜑=0 and 𝜑 satisfies−Δ𝜑=𝜑−𝐷₃+𝑐₂𝑢+𝑐₂𝑣−𝑐₃𝑤𝑑₃+𝑑₄𝑢⁻¹inΩ,𝜕_𝜂𝜑=0on𝜕Ω. (4.15) The Harnack inequality shows that min_Ω𝜑=0 implies 𝜑=0 on Ω. Hence, 𝑤=0 on Ω. From Lemma 4.5, we have (𝑢,𝑣)=(𝑢₁,𝑣₁). Define 𝑤_𝑗=𝑤_𝑗/‖𝑤_𝑗‖_∞; then, (𝑢_𝑗,𝑣_𝑗,𝑤_𝑗,𝑤_𝑗) satisfies −𝑑_1𝑗Δ𝑢_𝑗=𝑏₁𝑣_𝑗−𝑢_𝑗𝐷₁+𝑘𝑢_𝑗+𝑘𝑣_𝑗+𝑐₁𝑤_𝑗inΩ,−𝑑_2𝑗Δ𝑣_𝑗=𝑣_𝑗𝛽−𝑘𝑢_𝑗−𝑘𝑣_𝑗−𝑐₁𝑤_𝑗𝑑inΩ,−Δ_3𝑗𝑤_𝑗+𝑑_4𝑗u_𝑗𝑤_𝑗=𝑤_𝑗−𝐷₃+𝑐₂𝑢_𝑗+𝑐₂𝑣_𝑗−𝑐₃𝑤_𝑗𝜕inΩ,_𝜂𝑢_𝑗=𝜕_𝜂𝑣_𝑗=𝜕_𝜂𝑤_𝑗=0on𝜕Ω. (4.16)Similarly to the above, we can prove that there exists a subsequence of {𝑤_𝑗}, denoted by itself, and a nonnegative function 𝑤, such that 𝑤_𝑗→𝑤 in 𝐶²(Ω) and ‖𝑤‖_∞=1. Moreover, 𝑤 satisfies −𝑑₃+𝑑₄𝑢₁Δ𝑤=𝑤−𝐷₃+𝑐₂𝑢₁+𝑐₂𝑣₁inΩ,𝜕_𝜂𝑤=0on𝜕Ω. (4.17)Since ‖𝑤‖_∞=1, by the strong maximum principle and the Hopf boundary lemma, we find that 𝑤>0 on Ω. Applying the maximum principle again, we have −𝐷₃+𝑐₂𝑢₁+𝑐₂𝑣₁=0. Thus, 𝑢₁+𝑣₁=𝐷₃/𝑐₂. Noting that 𝑢₁+𝑣₁=𝛽/𝑘 in (1.2), it follows that 𝑐₂𝛽−𝑘𝐷₃=0, which is a contradiction to the condition 𝑅=𝑐₂𝛽−𝑘𝐷₃>0.
Next, we consider the remaining cases.
Integrating by parts, we obtain that𝑏_1Ω𝑣_𝑗𝑑𝑥=_Ω𝑢_𝑗𝐷₁+𝑘𝑢_𝑗+𝑘𝑣_𝑗+𝑐₁𝑤_𝑗𝑑𝑥,_Ω𝑣_𝑗𝛽−𝑘𝑢_𝑗−𝑘𝑣_𝑗−𝑐₁𝑤_𝑗𝑑𝑥=0,_Ω𝑤_𝑗−𝐷₃+𝑐₂𝑢_𝑗+𝑐₂𝑣_𝑗−𝑐₃𝑤_𝑗𝑑𝑥=0, (4.18)for 𝑗=1,2,…. Moreover, (𝑢,𝑣,𝑤) satisfies 𝑏_1Ω𝑣𝑑𝑥=_Ω𝑢𝐷₁+𝑘𝑢+𝑘𝑣+𝑐₁𝑤𝑑𝑥,_Ω𝑣𝛽−𝑘𝑢−𝑘𝑣−𝑐₁𝑤𝑑𝑥=0,_Ω𝑤−𝐷₃+𝑐₂𝑢+𝑐₂𝑣−𝑐₃𝑤𝑑𝑥=0. (4.19)
If 𝑑₁=∞, then 𝑢 satisfies−Δ𝑢=0inΩ,𝜕_𝜂𝑢=0on𝜕Ω. (4.20)Hence, 𝑢 is a constant. If 𝑢=0, from (4.19), we have in turn that 𝑣=𝑤=0. This contradicts Lemma 4.4. So, 𝑢 is a positive constant and either min_Ω𝑣=0 or min_Ω𝑤=0.
If 𝑑₂<∞. In the case of min_Ω𝑣=0, similarly to the arguments of (1), we have 𝑣=0 on Ω. This contradicts the first equation of (4.19). Thus min_Ω𝑣>0 and min_Ω𝑤=0. Note that 𝑤 satisfies−𝑑₃+𝑑₄𝑢Δ𝑤=𝑤−𝐷₃+𝑐₂𝑢−𝑐₃𝑤inΩ,𝜕_𝜂𝑤=0on𝜕Ω. (4.21)
If 𝑑₃<∞, the Harnack inequality implies that 𝑤=0 on Ω. If 𝑑₃=∞, then 𝑤 ia a constant. Since min_Ω𝑤=0, so 𝑤=0 on Ω. Therefore, by Lemma 4.5, (𝑢,𝑣,𝑤)=(𝑢₁,𝑣₁,0). Similarly to the arguments of (3), we arrive at 𝑐₂𝛽−𝑘𝐷₃=0, which is a contradiction.
Similarly, we can derive contradictions for all the other cases.

5. Existence of Stationary Patterns for the Model (1.5)

In this section we discuss the existence of nonconstant positive solutions to (2.1). These solutions are obtained for large cross-diffusion coefficient 𝑑₄, with the other parameters 𝑑₁, 𝑑₂, 𝑑₃, 𝑏₁, 𝐷₁, 𝐷₃, 𝑘, 𝑐₁, 𝑐₂, 𝑐₃, and 𝛽 suitably fixed. Our main result is as follows.

Theorem 5.1. Let the parameters 𝑑₁, 𝑑₂, 𝑑₃, 𝑏₁, 𝐷₁, 𝐷₃, 𝑘, 𝑐₁, 𝑐₂, 𝑐₃, and 𝛽 be fixed such that (H1), (H2), and (H3) hold. Let 𝜇 be given by the limit (2.13). If 𝜇∈(𝜇_𝑛,𝜇_𝑛+1) for some 𝑛≥2 and the sum ∑^𝑛_𝑖=2dim𝐸(𝜇_𝑖) is odd, then there exists a positive constant 𝑑^∗₄ such that (2.1) has at least one nonconstant positive solution for 𝑑₄>𝑑^∗₄.

Proof. By Proposition 2.2 and our assumption on 𝜇, there exists a positive constant 𝑑^∗₄ such that (2.14) holds if 𝑑₄>𝑑^∗₄, and 𝜇₁𝑑₄<0=𝜇₁<𝜇₂𝑑₄<𝜇₂,𝜇₃𝑑₄∈𝜇_𝑛,𝜇_𝑛+1. (5.1) We will prove that for any 𝑑₄>𝑑^∗₄, (2.1) has at least one nonconstant positive solution. The proof, which is by contradiction, is based on the homotopy invariance of the topological degree.
Suppose on the contrary that the assertion is not true for some 𝑑₄=𝑑₄>𝑑^∗₄. In the following we fix 𝑑₄=𝑑₄.
For 𝜃∈[0,1], define Φ(𝜃;𝐮)=(𝑑₁𝑢,𝑑₂𝑣,𝑑₃𝑤+𝜃𝑑₄𝑢𝑤)^𝑇 and consider the problem−Δ𝚽(𝜃;𝐮)=𝐆(𝐮)inΩ,𝜕_𝜂𝐮=0,on𝜕Ω. (5.2) Then 𝐮 is a positive nonconstant solution of (2.1) if and only if it is such a solution of (5.2) for 𝜃=1. It is obvious that 𝐮 is the unique constant positive solution of (5.2) for any 0≤𝜃≤1. As we observed in Section 2, for any 0≤𝜃≤1, 𝐮 is a positive solution of (5.2) if and only if 𝐅(𝜽;𝐮)∶=𝐮−(𝐈−Δ)⁻¹𝚽_𝐮⁻¹(𝜃;𝐮)𝐆(𝐮)+∇𝐮𝚽_𝐮𝐮(𝜃;𝐮)∇𝐮+𝐮=0in𝐘⁺. (5.3) It is obvious that 𝐅(1;𝐮)=𝐅(𝐮). Theorem 3.2 shows that 𝐅(0;𝐮)=0 has only the positive solution 𝐮 in 𝐘⁺. By a direct computation, 𝐷_𝐮𝐅(𝜃;𝐮)=𝐈−(𝐈−Δ)⁻¹𝚽_𝐮⁻¹(𝜃;𝐮)𝐆_𝐮(𝐮)+𝐈. (5.4)In particular, 𝐷_𝐮𝐅(0;𝐮)=𝐈−(𝐈−Δ)⁻¹D⁻¹𝐆_𝐮(𝐮)+𝐈, (5.5)where 𝐃=diag(𝑑₁,𝑑₂,𝑑₃) and 𝐷_𝐮𝐹(1;𝐮)=𝐈−(𝐈−Δ)⁻¹𝚽_𝐮⁻¹𝐆_𝐮(𝐮)+𝐈=𝐷_𝐮𝐅(𝐮). (5.6)From (2.5) and (2.9) we see that 𝐻𝜇=det𝚽_𝐮⁻¹(𝐮)𝒞𝑑₄;𝜇. (5.7)In view of (2.14) and (5.1), it follows that 𝐻𝜇₁(0)𝐻𝜇=𝐻>0,_𝑖𝐻𝜇<0,2≤𝑖≤𝑛,_𝑖>0,𝑖≥𝑛+1. (5.8)Therefore, zero is not an eigenvalue of the matrix 𝜇_𝑖𝐈−Φ_𝐮⁻¹(𝐮)𝐆_𝐮(𝐮) for all 𝑖≥1, and dim𝐸𝜇_{𝜇𝑖≥1,𝐻𝑖<0𝑖}=^𝑛_𝑖=2dim𝐸𝜇_𝑖=𝜎_𝑛,whichisodd. (5.9)Thanks to Proposition 2.1, we have index(𝐅(1;⋅),𝐮)=(−1)^𝛾=(−1)^𝜎𝑛=−1. (5.10) Similarly, we can easily show that index(𝐹(0;⋅),𝐮)=(−1)⁰=1. (5.11) Now, by Theorems 4.3 and 4.6, there exists a positive constant 𝐶 such that, for all 0≤𝜃≤1, the positive solutions of (2.1) satisfy 1/𝐶<𝑢,𝑣,𝑤<𝐶. Therefore, 𝐅(𝜃;𝐮)≠0 on 𝜕𝐵(𝐶) for all 0≤𝜃≤1. By the homotopy invariance of the topological degree, deg(𝐅(1;⋅),0,𝐵(𝐶))=deg(𝐅(0;⋅),0,𝐵(𝐶)). (5.12) On the other hand, by our supposition, both equations 𝐅(1;𝐮)=0 and 𝐅(0;𝐮)=0 have only the positive solution 𝐮 in 𝐵(𝐶). Hence, by (5.10) and (5.11), we have (𝐅()(𝐶(𝐅(),𝐮)(𝐅()(𝐶(𝐅(),𝐮)deg1;⋅,0,𝐵))=index1;⋅=−1,deg0;⋅,0,𝐵))=index0;⋅=1. (5.13) This contradicts (5.12), and thus we complete the proof of Theorem 5.1.

Remark 5.2. Assume that all the conditions hold in Theorem 5.1. Theorem 3.2 shows that 𝐮 is a globally asymptotically stable equilibrium for the system (1.4). However, Theorem 5.1 implies that the cross-diffusion system (1.5) has at least one nonconstant positive steady state. Our results demonstrate that stationary patterns can be found due to the emergence of cross-diffusion.

6. Nonexistence of Nonconstant Positive Solution of (2.1)

In this section, we discuss the nonexistence of nonconstant positive solution of (2.1) when the cross-diffusion coefficient 𝑑₄>0 is small.

Theorem 6.1. If the parameters 𝑑₁, 𝑑₃, 𝑑₄, 𝑏₁, 𝐷₁, 𝐷₃, 𝑘, 𝑐₁, 𝑐₂, 𝑐₃, and 𝛽 satisfy (H1), (H3), and 𝑐₁𝑑²₄𝑢𝑤𝑐₂<4𝑑₁𝑑₃, (H4) then the problem (2.1) has no nonconstant positive solution.

Proof. Assume that (𝑢,𝑣,𝑤) is a positive solution of (2.1). Let 𝜆=𝑢, 𝜌=𝑐₁𝑢/𝑐₂. Multiplying the equations of (2.1) by (𝑢−𝑢), 𝜆(𝑣−𝑣)/𝑣, and 𝜌(𝑤−𝑤)/𝑤, respectively, and integrating by parts, as in the proof of Theorem 3.2, we obtain 0=−𝐼₃−𝐼₄, where 𝐼₃=_Ω𝑑₁|∇𝑢|²𝑑+𝜆²𝑣𝑣²|∇𝑣|²𝑑+𝜌𝑤₃+𝑑₄𝑢𝑤²|∇𝑤|²𝑑+𝜌⁴𝑤𝑤𝐼∇𝑢⋅∇𝑤𝑑𝑥,₄=_Ω𝐷₁+𝑘𝑢+𝑘𝑢+𝑘𝑣+𝑐₁𝑤(𝑢−𝑢)²+𝜆𝑘(𝑣−𝑣)²+𝜌𝑐₃(𝑤−𝑤)²+2𝑘𝑢−𝑏₁(𝑢−𝑢)(𝑣−𝑣)𝑑𝑥. (6.1)Applying (H3) and (H4), it is easy to prove that 𝐼₃≥0 and 𝐼₄≥0. This implies that (𝑢,𝑣,𝑤)=(𝑢,𝑣,𝑤) on Ω and the proof is complete.

Remark 6.2. Theorem 6.1 shows that the problem (2.1) has no nonconstant positive solutions if one of 𝑑₁ and 𝑑₃ is sufficiently large; that is, unlimitedly increasing one of the diffusion rates 𝑑₁ and 𝑑₃ will eventually wipe out all nonconstant solutions of (2.1). However, Theorem 6.1 does not tell us the effect of the diffusion rate 𝑑₂ on the stationary problem (2.1). Using the similar arguments in Section 2, we can find that 𝑑₂ does not cause instability of 𝐮. Therefore, we conjecture that the problem (2.1) has no nonconstant positive solutions if 𝑑₂ is sufficiently large.

Remark 6.3. Theorems 5.1 and 6.1 seem to indicate that diffusion tends to suppress pattern formation, while cross-diffusion seems to help create patterns.

7. Discussion

In this paper, we have introduced a more realistic mathematical model for a diffusive predator-prey system where the prey has a sex structure comprising male and female members. In this model, we model the tendency of the predators to keep away from the male prey by a cross-diffusion. As a result, our model is a strongly coupled cross-diffusion system, which is mathematically more complex than systems used to model sex-structured predator-prey behavior hitherto [1, 8]. What is noteworthy about this model is that, as the cross-diffusion term arises, it is precisely this cross-diffusion that destabilizes the uniform positive equilibrium and gives rise to stationary patterns for the model. Indeed, stationary patterns do not arise for the ODE (spatially independent) model, nor the PDE model without cross-diffusion.

In fact, one can see that this particular cross-diffusion term is also significant from the mathematical point of view. The following system represents the general form of SKT-type cross-diffusion [22] in the predator-prey model𝑢_𝑡−Δ𝑑₁𝑢+𝑑₁₂𝑢𝑣+𝑑₁₃𝑢𝑤=𝐺₁()𝑣𝑢,𝑣,𝑤,𝑥∈Ω,𝑡>0,_𝑡−Δ𝑑₂𝑣+𝑑₂₁𝑢𝑣+𝑑₂₃𝑣𝑤=𝐺₂()𝑤𝑢,𝑣,𝑤,𝑥∈Ω,𝑡>0,_𝑡−Δ𝑑₃𝑤+𝑑₃₁𝑢𝑤+𝑑₃₂𝑣𝑤=𝐺₃()𝜕𝑢,𝑣,𝑤,𝑥∈Ω,𝑡>0,_𝜂𝑢(𝑥,𝑡)=𝜕_𝜂𝑣(𝑥,𝑡)=𝜕_𝜂𝑤()𝑢()𝑥,𝑡=0,𝑥∈𝜕Ω,𝑡>0,𝑥,0=𝑢₀(𝑥),𝑣(𝑥,0)=𝑣₀(𝑥),𝑤(𝑥,0)=𝑤₀(𝑥),𝑥∈Ω, (7.1) where 𝑑_𝑖𝑗, 𝑖,𝑗=1,2,3, 𝑖≠𝑗, are cross-diffusion coefficients. Using the method in Section 2 of this paper, we have investigated the stability of 𝐮 for each 𝑑_𝑖𝑗, respectively. We found that either 𝑑₁₃ or 𝑑₂₃ does not cause instability of 𝐮, while each of the other 𝑑_𝑖𝑗 can induce the instability of 𝐮. For 𝑑₃₂≠0, we can obtain the similar conclusions as Theorem 5.1 by the same mathematical treatment in this paper. Unfortunately, the existence of nonconstant positive steady states has not been obtained when 𝑑₁₂≠0 or 𝑑₂₁≠0, because we cannot establish a priori lower bounds for all possible positive steady states of (7.1).

On the other hand, as pointed out in [23], a Lotka-Volterra-type model can be regarded as a local approximation to a nonlinear system. In the present paper, we only consider the case that the interaction terms on the right-hand side of (1.5) are linear. For the nonlinear case, it will be extremely difficult to analyze positive steady states.

In this paper, we do not discuss the stability and the number of the nonconstant positive solutions. We will consider them in the coming papers.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (no. 11061031, 11101334), the NSF of Gansu Province (096RJZA118), the Fundamental Research Funds for the Gansu University, and NWNU-KJCXGC-03-47, 61 Foundations.