Uniqueness and Multiplicity of a Prey-Predator Model with Predator Saturation and Competition

Abstract

We investigate positive solutions of a prey-predator model with predator saturation and competition under homogeneous Dirichlet boundary conditions. First, the existence of positive solutions and some sufficient and necessary conditions is established by using the standard fixed point index theory in cones. Second, the changes of solution branches, multiplicity, uniqueness, and stability of positive solutions are obtained by virtue of bifurcation theory, perturbation theory of eigenvalues, and the fixed point index theory. Finally, the exact number and type of positive solutions are proved when 𝑘 or 𝑚 converges to infinity.

1. Introduction

Considering the destabilizing force of predator saturation and the stabilizing force of competition for prey, Bazykin [1] proposed the function response 𝑓(𝑢,𝑣)=1/(1+𝑚𝑢)(1+𝑘𝑣) in the prey-predator model instead of the classical Holling-type II functional response. For this functional response, the prey-predator model is taken as the following form:𝑢_𝑡−Δ𝑢=𝑢𝑎−𝑢−𝑏𝑣()𝑣1+𝑚𝑢)(1+𝑘𝑣,𝑥∈Ω,𝑡>0,_𝑡−Δ𝑣=𝑣𝑐−𝑣+𝑑𝑢()𝑢()1+𝑚𝑢)(1+𝑘𝑣,𝑥∈Ω,𝑡>0,𝑢=𝑣=0,𝑥∈𝜕Ω,𝑡>0,𝑥,0=𝑢₀(𝑥)≥0,𝑣(𝑥,0)=𝑣₀(𝑥)≥0,𝑥∈Ω. (1.1)

In this paper, we are concerned with the positive solution of the boundary value problem of the following elliptic system corresponding to the system (1.1):−Δ𝑢=𝑢𝑎−𝑢−𝑏𝑣(1+𝑚𝑢)(1+𝑘𝑣),𝑥∈Ω,−Δ𝑣=𝑣𝑐−𝑣+𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣),𝑥∈Ω,𝑢=𝑣=0,𝑥∈𝜕Ω, (1.2) where Ω is a bounded domain in 𝑅^𝑁(𝑁≥1) with smooth boundary 𝜕Ω;  𝑎,𝑏,𝑐 are positive constants; 𝑑 is constant; 𝑚 and 𝑘 are nonnegative constants.

If 𝑚=𝑘=0, then (1.2) is reduced to the classical Lotka-Volterra prey-predator model which has received extensive study in the last decade, see [2–8]. In particular, the existence of positive solutions for this case was completely understood, see Dancer [8]. It has been conjectured that there is at most one positive solution, but this was shown only for the case the space dimension 𝑛 is one, see [9]. For space dimension greater than one, this is still an open problem; we also refer to [10, 11] for some partial results on uniqueness. The stability of positive solutions was studied in [10, 11], but the results are still far from being complete.

The case when 𝑚>0 and 𝑘=0 was first studied by Blat and Brown [12]. In this case, the term 𝑓(𝑢,𝑣) is known as the Holling-Tanner interaction term, and we refer to [5, 12–17] for more discussion on this model. In [12], Blat and Brown studied the existence of positive solutions to (1.2) by making use of both local and global bifurcation theories. The case when 𝑚 goes to infinity was extensively studied by Du and Lou in [13, 14, 18]. They gave a good understanding of the existence, stability, and number of positive solutions for large 𝑚.

However, the case when 𝑚>0 and 𝑘>0 was first studied by Bazykin in the paper [1], more detailed background on this case, we can refer to [1]. And more works can refer to [19], Wang studies the existence, multiplicity, and stability of positive solutions of (1.2). However, Our work is more specific and meticulous than theirs. In particular. Firstly, the changes of solution branches, uniqueness, and stability of positive solutions are obtained by virtue of bifurcation theory, perturbation theory of eigenvalues, and the fixed point index theory. Secondly, the exact number and type of positive solutions are proved when 𝑘 or 𝑚 is large.

This paper is organized as follows: in Section 2, we give sufficient and necessary conditions for the existence of coexistence states of (1.2) by using index theory. In Section 3, by using 𝑎 as a main bifurcation parameter, the multiplicity of coexistence stats to (1.2) is investigated in the gap between the sufficient and necessary conditions for the existence of coexistence states which are found in Section 2. In Section 4, the multiplicity, uniqueness, and stability of coexistence states of (1.2) are investigated when 𝑘 or 𝑚 converges to infinity.

2. Existence and Nonexistence of Coexistence States

In this section, we will obtain existence and nonexistence of coexistence states. Firstly, we present some basic results which will be used in this paper.

Let 𝜆₁(𝑞)<𝜆₂(𝑞)≤𝜆₃(𝑞)≤⋯ be all eigenvalues of the following problem: −Δ𝜙+𝑞(𝑥)𝜙=𝜆𝜙,𝜙_𝜕Ω=0, (2.1) where 𝑞(𝑥)∈𝐶(Ω). It is easy to see that 𝜆₁(𝑞) is simple and 𝜆₁(𝑞) is strictly increasing in the sense that 𝑞₁≤𝑞₂ and 𝑞₁≢𝑞₂ imply 𝜆₁(𝑞₁)<𝜆₁(𝑞₂). When 𝑞(𝑥)≡0, we denote by 𝜆_𝑖𝜆_𝑖(0). Moreover, we denote by Φ₁ the eigenfunction corresponding to 𝜆₁ with normalization ‖Φ₁‖_∞=1 and positive in Ω.

Define 𝐶₀(Ω)={𝑢∈𝐶(Ω)∣𝑢=0on𝜕Ω}. It is well known that for any 𝑎>𝜆₁, the problem −Δ𝑢=𝑎𝑢−𝑢²,𝑢|_𝜕Ω=0 (2.2) has a unique positive solution which we denote by 𝜃_𝑎. It is well known that the mapping 𝑎→𝜃_𝑎 is strictly increasing, continuously differentiable from (𝜆₁,∞) to 𝐶²(Ω)∩𝐶₀(Ω) and that 𝜃_𝑎→0 uniformly on Ω as 𝑎→𝜆₁. Moreover, we have 0<𝜃_𝑎<𝑎 in Ω. It follows that (1.2) has two semitrivial solutions (𝜃_𝑎,0) and (0,𝜃_𝑐) if 𝑎,𝑐>𝜆₁.

Next, we give an a priori estimate based on maximum principle. Its proof will be omitted.

Lemma 2.1. Any coexistence state (𝑢,𝑣) of (1.2) has an a priori boundary, that is, 𝑢≤𝑎,𝑣≤𝐵∶=𝑐+𝑎𝑑.1+𝑎𝑚 (2.3)

In the following, we set up the fixed point index theory for later use. Let 𝐸 be a Banach space. 𝑊⊂𝐸 is called a wedge if 𝑊 is a closed convex set and 𝛽𝑊⊂𝑊 for all 𝛽≥0. For 𝑦∈𝑊, we define 𝑊_𝑦={𝑥∈𝐸∶∃𝑟=𝑟(𝑥)>0,s.t.𝑦+𝑟𝑥∈𝑊}, 𝑆_𝑦={𝑥∈𝑊_𝑦∶−𝑥∈𝑊_𝑦}∶ we always assume that 𝐸=𝑊−𝑊. Let 𝑇∶𝑊_𝑦→𝑊_𝑦 be a compact linear operator on 𝐸. We say that 𝑇 has property 𝛼 on 𝑊_𝑦 if there exists 𝑡∈(0,1) and 𝜔∈𝑊_𝑦⧵𝑆_𝑦, such that 𝜔−𝑡𝑇𝜔∈𝑆_𝑦.

For any 𝛿>0 and 𝑦∈𝑊, we denote 𝐵⁺_𝛿(𝑦)=𝐵_𝛿(𝑦)∩𝑊. Assume that 𝐹∶𝐵⁺_𝛿(𝑦)→𝑊 is a compact operator and 𝑦 is an isolated fixed point of 𝐹. If 𝐹^′ is Fréchet differentiable at 𝑦, then the derivative 𝐹^′(𝑦) has the property that 𝐹^′(𝑦)∶𝑊_𝑦→𝑊_𝑦. We denote by index_𝑊(𝐹,𝑦) the fixed point index of 𝐹 at 𝑦 relative to 𝑊.

We state a general result of Dancer [20] on the fixed point index with respect to the positive cone 𝑊 (see also [6]).

Lemma 2.2. Suppose that 𝐼−𝐿 is invertible on 𝑊_𝑦.(i) If 𝐿 has property 𝛼 on 𝑊_𝑦, then index_𝑊(𝐹,𝑦)=0.(ii) If 𝐿 does not have property 𝛼 on 𝑊_𝑦, then index_𝑊(𝐹,𝑦)=(−1)^𝜎, where 𝜎 is the sum of algebra multiplicities of the eigenvalues of 𝐿 which are greater than 1.

We introduce some notations as follows: 𝑋=𝐶¹₀(Ω)⨁𝐶¹₀(Ω), where 𝐶¹₀(Ω)={𝜔∈𝐶¹(Ω)∶𝜔∣_𝜕Ω=0}, 𝑊=𝐾⨁𝐾, where 𝐾={𝜑∈𝐶(Ω)∶𝜑(𝑥)≥0}, 𝐷∶={(𝑢,𝑣)∈𝑋∶𝑢≤𝑎+1,𝑣≤𝐵+1}, 𝐷^′∶=(int𝐷)⋂𝑊.

Define 𝐹_𝑡∶𝐷^′→𝑊 by 𝐹_𝑡(𝑢,𝑣)=(−Δ+𝑃)⁻¹𝑡𝑢𝑎−𝑢−𝑏𝑣(1+𝑚𝑢)(1+𝑘𝑣)+𝑃𝑢𝑡𝑣𝑐−𝑣+𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣)+𝑃𝑣, (2.4) where 𝑡∈[0,1] and 𝑃>max{𝑎+𝑏𝐵,𝑐+𝑑𝑎/(1+𝑚𝑎)}. It follows from Maximum Principle that (−Δ+𝑃)⁻¹ is a compact positive operator, 𝐹_𝑡 is complete continuous and Fréchet differentiable. Denote 𝐹₁=𝐹, observe that (1.2) has a positive solution in 𝑊 if and only if 𝐹_𝑡=𝐹₁ has a positive fixed point in 𝐷^′.

If 𝑎>𝜆₁ and 𝑐>𝜆₁, then (0,0), (𝜃_𝑎,0), and (0,𝜃_𝑐) are the only nonnegative fixed points of 𝐹. Then index_𝑊(𝐹,(0,0)), index_𝑊(𝐹,(𝜃_𝑎,0)), and index_𝑊(𝐹,(0,𝜃_𝑐)) are well defined. We calculate the Fréchet operator of 𝐹 as follows: (−Δ+𝑃)⁻¹𝑎−2𝑢−𝑏𝑣(1+𝑚𝑢)²(1+𝑘𝑣)+𝑃−𝑏𝑢(1+𝑚𝑢)(1+𝑘𝑣)²𝑑𝑣(1+𝑚𝑢)²(1+𝑘𝑣)𝑐−2𝑣+𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣)²+𝑃. (2.5)

We can obtain the following lemmas by similar methods to those in the proofs of Lemmas 1 and 2 in [19].

Lemma 2.3. Suppose that 𝑎>𝜆₁, one has(i)deg_𝑊(𝐼−𝐹,𝐷^′)=1, where deg_𝑊(𝐼−𝐹,𝐷^′) is the degree of 𝑇−𝐹 in 𝐷^′ relative to 𝑊(ii) if 𝑐≠𝜆₁, then index_𝑊(𝐹,(0,0))=0;(iii) if 𝑐>𝜆₁(−𝑑𝜃_𝑎/(1+𝑚𝜃_𝑎)), then index_𝑊(𝐹,(𝜃_𝑎,0))=0;(iv) if 𝑐<𝜆₁(−𝑑𝜃_𝑎/(1+𝑚𝜃_𝑎)), then index_𝑊(𝐹,(𝜃_𝑎,0))=1.

Lemma 2.4. Suppose that 𝑐>𝜆₁, one has(i) if 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), then index_𝑊(𝐹,(0,𝜃_𝑐))=0;(ii) if 𝑎<𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), then index_𝑊(𝐹,(0,𝜃_𝑐))=1.

Next, we will show some results of existence and nonexistence of positive solutions of (1.2).

Theorem 2.5. (𝑖) If 𝑎≤𝜆₁, then (1.2) has no positive solution; if 𝑎≤𝜆₁ and 𝑐≤𝜆₁, then (1.2) has no nonnegative nonzero solution.
(𝑖𝑖) If 𝑐≤𝜆₁ and (1.2) has a positive solution, then 𝑎>𝜆₁,𝑐+𝑑𝑎/(1+𝑚𝑎)>𝜆₁.
(𝑖𝑖𝑖) If 𝑐>𝜆₁ and (1.2) has a positive solution, then 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑚𝜃_𝑎)(1+𝑘𝜃_𝑐)).

Proof. (i) First assume that (𝑢,𝑣) is a positive solution of (1.2), then (𝑢,𝑣) satisfies −Δ𝑢=𝑢𝑎−𝑢−𝑏𝑣(1+𝑚𝑢)(1+𝑘𝑣),𝑥∈Ω,𝑢=0,𝑥∈𝜕Ω, (2.6) and so 𝑎=𝜆₁(𝑢+𝑏𝑣/(1+𝑚𝑢)(1+𝑘𝑣)) by the eigenvalue problem. Due to the comparison principle for eigenvalues, we have 𝑎>𝜆₁, a contradiction. Next, assume that (𝑢,𝑣) is a nonnegative nonzero solution of (1.2). If 𝑢≢0 and 𝑣≡0, then 𝑎>𝜆₁ by the previous proof. We can also similarly derive 𝑐>𝜆₁ when 𝑢≡0 and 𝑣≢0, which is a contradiction again.
(ii) Assume that (𝑢,𝑣) is a positive solution of (1.2). Then 𝑎>𝜆₁ by (i), and so the positive semitrivial solution 𝜃_𝑎 exists. Since −Δ𝑢=𝑢𝑎−𝑢−𝑏𝑣()()1+𝑚𝑢)(1+𝑘𝑣≤𝑢𝑎−𝑢,𝑥∈Ω,𝑢=0,𝑥∈𝜕Ω, (2.7)𝑢 is a lower solution of (1.2). By the uniqueness of 𝜃_𝑎, 𝑢≤𝜃_𝑎. Furthermore, since 𝑣 satisfies the equation −Δ𝑣=𝑣𝑐−𝑣+𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣),𝑥∈Ω,𝑣=0,𝑥∈𝜕Ω, (2.8) one has 0=𝜆₁(−𝑐+𝑣−𝑑𝑢/(1+𝑚𝑢)(1+𝑘𝑣))>𝜆₁(−𝑐−𝑑𝑎/(1+𝑚𝑎)), which implies the result.
(iii) Let (𝑢,𝑣) be a positive solution of (1.2); then 𝜃_𝑎 exists with 𝑢≤𝜃_𝑎 as in (ii). Similarly, the given assumption 𝑐>𝜆₁ implies the existence of positive solution 𝜃_𝑐 of (1.2) with 𝜃_𝑐≤𝑣. Similar to the proof of (i), we have 𝑎=𝜆₁(𝑢+𝑏𝑣/(1+𝑚𝑢)(1+𝑘𝑣))>𝜆₁(𝑏𝜃_𝑐/(1+𝑚𝜃_𝑎)(1+𝑘𝜃_𝑐)). This follows since the function 𝑏𝑣/(1+𝑚𝑢)(1+𝑘𝑣) has a minimum at 𝑢=𝜃_𝑎 and 𝑣=𝜃_𝑐 (for 𝑢≤𝜃_𝑎 and 𝑣≥𝜃_𝑐).

Theorem 2.6. (𝑖) If 𝑐>𝜆₁ and 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). Then (1.2) has at least a positive solution.
(𝑖𝑖) Suppose that 𝑐<𝜆₁. Then (1.2) has positive solution if and only if 𝑎>𝜆₁ and 𝑐>𝜆₁(−𝑑𝜃_𝑎/(1+𝑚𝜃_𝑎)).

Proof. (i) By Lemmas 2.3 and 2.4, we have deg_𝑊(𝐼−𝐹,𝐷)−index_𝑊𝑓,(0,0)−index_𝑊𝑓,𝜃_𝑎,0−index_𝑊𝑓,0,𝜃_𝑐=1. (2.9) So (1.2) has at least one positive solution.
(ii) We first prove the sufficiency. Since 𝑐<𝜆₁, (1.2) has no solution taking the form (0,𝑣) with 𝑣>0. If 𝑎>𝜆₁ and 𝑐>𝜆₁(−𝑑𝜃_𝑎/(1+𝑚𝜃_𝑎)), note that 𝑐<𝜆₁; from Lemma 2.4, we have deg_𝑊(𝐼−𝐹,𝐷)−index_𝑊𝑓,(0,0)−index_𝑊𝑓,𝜃_𝑎,0=1. (2.10) Hence (1.2) has at least one positive solution.
Conversely, suppose that (𝑢,𝑣) is a positive solution of (1.2). Then 𝑎>𝜆₁, and 𝑢<𝜃_𝑎. Since (𝑢,𝑣) satisfies −Δ𝑣=𝑣(𝑐−𝑣+𝑑𝑢/(1+𝑚𝑢)(1+𝑘𝑣)),𝑥∈Ω,𝑣=0,𝑥∈𝜕Ω. (2.11) It follows that 0=𝜆₁(−𝑐+𝑣−𝑑𝑢/(1+𝑚𝑢)(1+𝑘𝑣))>𝜆₁(−𝑐−𝑑𝑎/(1+𝑚𝑎)).

Theorem 2.7. If one of the following conditions holds, then (1.2) has no positive solutions:(i)𝑏≥(1+𝑚𝑎)(1+𝑘𝐵) and 𝑎≤𝑐,(ii)𝑏<(1+𝑚𝑎)(1+𝑘𝐵) and 𝑐−𝑎≥(1−𝑏𝑎/(1+𝑚𝑎)(1+𝑘𝐵))𝐵.

Proof. Since the proof of Theorem 2.7 is similar to the proof of Theorem 3 of [19], we omit it.

3. Global Bifurcation and Stability of Positive Solution

In this section, we consider a positive solution bifurcates from the semitrivial nonnegative branch {(0,𝜃_𝑐,𝑎)} by taking 𝑎 as a bifurcation parameter and fixing 𝑐>𝜆₁. Furthermore, we show that the existence of global bifurcation of (1.2) with respect to parameter 𝑎 and its stability. Moreover, the multiplicity, uniqueness, and stability of positive solutions are obtained by means of perturbation theory of eigenvalues and the fixed point theory.

Let 𝑎 be the principal eigenvalue of the following problem: −Δ𝜙+𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜙=𝑎𝜙,𝜙_𝜕Ω=0, (3.1) and Φ is the corresponding eigenfunction with ‖Φ‖_∞=1.

Let 𝜔=𝑢,𝜒=𝑣−𝜃_𝑐; then 0≤𝜔≤𝜃_𝑎,𝜒≥0, and 𝜔,𝜒 satisfies −Δ𝜔=𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜔+𝐹₁𝜔,𝜒,𝑥∈Ω,−Δ𝜒=𝑐−2𝜃_𝑐𝜒+𝑑𝜃_𝑐1+𝑘𝜃_𝑐𝜔+𝐹₂𝜔,𝜒,𝑥∈Ω,𝜔=𝜒=0,𝑥∈𝜕Ω, (3.2) where 𝐹₁𝜔,𝜒=𝑏𝜔𝜃_𝑐1+𝑘𝜃_𝑐−𝑏𝜔𝜒+𝜃_𝑐(1+𝑚𝜔)1+𝑘𝜒+𝜃_𝑐−𝜔²,𝐹₂𝜔,𝜒=𝑑𝜔𝜒+𝜃_𝑐(1+𝑚𝜔)1+𝑘𝜒+𝜃_𝑐−𝑑𝜔𝜃_𝑐1+𝑘𝜃_𝑐−𝜒². (3.3)

Clearly, 𝐹=(𝐹₁,𝐹₂) is continuous, 𝐹(0,0)=0, and the Fréchet derivative 𝐷_(𝜔,𝜒)𝐹∣_(0,0)=0. Let 𝐾 be the inverse of −Δ with Dirichlet boundary condition. Then, we have 𝜔=𝑎𝐾𝜔−𝑏𝐾𝜔𝜃_𝑐1+𝑘𝜃_𝑐+𝐾𝐹₁𝜔,𝜒,𝑥∈Ω,𝜒=𝑐𝐾𝜒−2𝐾𝜒𝜃_𝑐+𝑑𝐾𝜔𝜃_𝑐1+𝑘𝜃_𝑐+𝐾𝐹₂𝜔,𝜒,𝑥∈Ω,𝜔=𝜒=0,𝑥∈𝜕Ω. (3.4)

Define the operator 𝑇∶𝑅⁺×𝑋→𝑋 as follows: 𝑇𝑎;𝜔,𝜒=𝑎𝐾𝜔−𝑏𝐾𝜔𝜃_𝑐1+𝑘𝜃_𝑐+𝐾𝐹₁𝜔,𝜒𝑐𝐾𝜒−2𝐾𝜒𝜃_𝑐+𝑑𝐾𝜔𝜃_𝑐1+𝑘𝜃_𝑐+𝐾𝐹₂𝜔,𝜒, (3.5) then 𝑇(𝑎;𝜔,𝜒) is a compact operator on 𝑋. Let 𝐺(𝑎;𝜔,𝜒)=(𝜔,𝜒)^𝑇−𝑇(𝑎;𝜔,𝜒); then 𝐺 is continuous, and 𝐺(𝑎;0,0)=0. 𝐺(𝑎;𝜔,𝜒)=0 with 0≤𝜔≤𝜃_𝑎,𝜒≥0 if and only if (𝜔,𝜒+𝜃_𝑐,𝑎) is a nonnegative solution of (1.2).

Lemma 3.1. Assume that 𝑐>𝜆₁. Then (𝑎;𝑢,𝑣)=(𝑎;0,𝜃_𝑐) is a bifurcation point of (3.2), and there exist positive solutions of (3.2) in the neighborhood of (𝑎;0,𝜃_𝑐), where 𝑎=𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)).

Remark 3.2. The proof of Lemma 3.1 is similar to the proof of Theorem 9 in [19]. The proof of Lemma 3.1 shows that there exist 𝛿>0 and 𝐶¹ continuous curve (𝑎(𝑠);𝜙(𝑠),𝜑(𝑠))∶(−𝛿,𝛿)→𝑅×𝑍 such that 𝑎(0)=𝑎,𝜙(0)=0,𝜑(0)=0, and (𝑎(𝑠);𝜔(𝑠),𝜒(𝑠))=(𝑎(𝑠);𝑠(Φ+𝜙(𝑠)),𝑠(Ψ+𝜑(𝑠))) satisfies 𝐺(𝑎(𝑠);𝜔(𝑠),𝜒(𝑠))=0, where 𝑋=𝑍⊕span{(Φ,Ψ)}. Hence (𝑎(𝑠);𝑈(𝑠),𝑉(𝑠))(|𝑠|<𝛿) is a bifurcation solution of (3.2), where 𝑈(𝑠)=𝑠(Φ+𝜙(𝑠)),𝑉(𝑠)=𝜃_𝑐+𝑠(Ψ+𝜑(𝑠)), Ψ=(−Δ−𝑐+2𝜃_𝑐)⁻¹(𝑑𝜃_𝑐/(1+𝑘𝜃_𝑐)Φ).
If we take 0<𝑠<𝛿, then the nontrivial nonnegative solutions of (1.2) close to (𝑎;0,𝜃_𝑐) are either on the branch {(𝑎;0,𝜃_𝑐)∶𝑎∈𝑅⁺} or the branch {(𝑎(𝑠);𝑈(𝑠),𝑉(𝑠))∶0<𝑠<𝛿}.
Let 𝑇∶𝑋×𝑅→𝑋 be a compact continuously differentiable operator such that 𝑇(0,𝑎)=0. Suppose that we can write 𝑇 as 𝑇(𝑢,𝑎)=𝐾(𝑎)𝑢+𝑅(𝑢,𝑎), where 𝐾(𝑎) is a linear compact operator and the Fréchet derivative 𝑅_𝑢(0,0)=0. If 𝑥₀ is an isolated fixed point of 𝑇, then we can define the index if 𝑇 at 𝑥₀ as index(𝑇,𝑥₀)=deg(𝐼−𝑇,𝑈_𝛿(𝑥₀),𝑥₀), where 𝑈_𝛿(𝑥₀) is a ball with center at 𝑥₀ such that 𝑥₀ is the only fixed point of 𝑇 in 𝑈_𝛿(𝑥₀). If 𝐼−𝑇^′(𝑥₀) is invertible, then 𝑥₀ is an isolated fixed point of 𝑇 and index(𝑇,𝑥₀)=deg(𝐼−𝑇,𝑈_𝛿(𝑥₀),𝑥₀)=deg(𝐼−𝑇^′(𝑥₀),𝑈_𝛿(𝑥₀),0). If 𝑥₀=0, then it is well known that the Leray-Schauder degree deg(𝐼−𝐾(𝑎),𝑈_𝛿(𝑥₀),0)=(−1)^𝜎, where 𝜎 is equal to the algebraic multiplicities of the eigenvalue of 𝐾 which is greater than one.
Next, we will extend the local bifurcation solution {(𝑎(𝑠);𝑈(𝑠),𝑉(𝑠))∶0<𝑠<𝛿} given by Lemma 3.1 to the global bifurcation.
Let 𝑃₁={𝑢∈𝐶¹₀(Ω)∶𝑢(𝑥)>0,𝑥∈Ω,𝜕𝑢/𝜕𝑛<0,𝑥∈𝜕Ω},𝑃={(𝑢,𝑣,𝑎)∈𝑋×𝑅⁺∶𝑢,𝑣∈𝑃₁}.

Theorem 3.3. Suppose that 𝑐>𝜆₁; then the global bifurcation 𝐶 of bifurcating branch of positive solutions of (1.2) becomes unbounded by 𝑎 going to infinity in 𝑃.

Proof. Let 𝑇^′(𝑎)⋅𝜔,𝜒=𝐷_(𝜔,𝜒)𝑇()⋅=𝑎;0,0𝜔,𝜒𝑎𝐾𝜔−𝑏𝐾𝜔𝜃_𝑐1+𝑘𝜃_𝑐,𝑐𝐾𝜒−2𝐾𝜒𝜃_𝑐+𝑑𝐾𝜔𝜃_𝑐1+𝑘𝜃_𝑐. (3.6) Suppose that 𝜇≥1 is an eigenvalue of 𝑇^′(𝑎). Then we have −𝜇Δ𝜔=𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜔,𝑥∈Ω,−𝜇Δ𝜒=𝑐−2𝜃_𝑐𝜒+𝑑𝜃_𝑐1+𝑘𝜃_𝑐𝜔,𝑥∈Ω,𝜔=𝜒=0,𝑥∈𝜕Ω. (3.7) Clearly 𝜔≢0, otherwise, 𝜔≡0, since all eigenvalue of the operator (−𝜇Δ−𝑐+2𝜃_𝑐) is greater than 0, so 𝜒≡0, a contradiction. Therefore, for some 𝑖 such that 𝑎=𝑎_𝑖(𝜇) is the eigenvalue of the following problem: −𝜇Δ𝜔+𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜔=𝑎𝜔,𝜔|_𝜕Ω=0. (3.8) It is well known that 𝑎_𝑖(𝜇) is increasing with respect to 𝜇 on [1,+∞) and can be ordered as 0<𝑎₁𝜇<𝑎₂𝜇≤𝑎₃𝜇≤⋯→∞,𝑎₁(1)=𝑎. (3.9) On the other hand, if 𝜇≥1, then all eigenvalues of (−𝜇Δ−𝑐+2𝜃_𝑐) are greater than 0; furthermore, 𝜒=(−𝜇Δ−𝑐+2𝜃_𝑐)⁻¹(𝑑𝜃_𝑐/(1+𝑘𝜃_𝑐)𝜔). Thus, 𝜇≥1 is the eigenvalue of 𝑇^′(𝑎) if and only if there exists some 𝑖, such that 𝑎=𝑎_𝑖(𝜇).
Suppose that 𝑎<𝑎. Then for any 𝜇≥1,𝑖≥1,𝑎<𝑎₁(1)≤𝑎_𝑖(𝜇). Hence, 𝑇^′(𝑎) has no eigenvalue greater than 1, and index(𝑇(𝑎;⋅),0)=1 as 𝑎<𝑎.
Suppose that 𝑎<𝑎<𝑎₂(1). Then for any 𝜇≥1,𝑖≥2,𝑎<𝑎_𝑖(𝜇). Since 𝑎₁(1)=𝑎, lim_𝜇→∞𝑎₁(𝜇)=+∞, and 𝑎₁(𝜇) is increasing with respect to 𝜇. Hence, there exists a unique 𝜇₁>1, such that 𝑎=𝑎₁(𝜇₁). So 𝑁(𝜇₁𝐼−𝑇^′(𝑎))=span{(𝜔,𝜒)},dim𝑁(𝜇₁𝐼−𝑇^′(𝑎))=1, where 𝜔>0 is the principal eigenvalue of the following problem: 𝜇₁Δ𝜔+𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜔=0,𝜔|_𝜕Ω=0, (3.10) where 𝜒=(−𝜇₁Δ−𝑐+2𝜃_𝑐)⁻¹(𝑑𝜃_𝑐/(1+𝑘𝜃_𝑐)𝜔).
In the following, we will prove that 𝑅(𝜇₁𝐼−𝑇^′(𝑎))∩𝑁(𝜇₁𝐼−𝑇^′(𝑎))=0. In fact, if the assertion is false, we may assume that (𝜔,𝜒)∈𝑅(𝜇₁𝐼−𝑇^′(𝑎)). Then there exists (𝜔,𝜒)∈𝑋, such that (𝜇₁𝐼−𝑇^′(𝑎))(𝜔,𝜒)=(𝜔,𝜒), that is, 𝜇₁Δ𝜔+𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜔=Δ𝜔,𝜔|_𝜕Ω=0. (3.11) Multiplying the equation by 𝜔, integrating over Ω, and using Green's formula, we obtain _Ω𝜔Δ𝜔=_Ω𝜇₁Δ𝜔+𝑎𝜔−𝑏𝜔𝜃_𝑐1+𝑘𝜃_𝑐𝜔=_Ω𝜇₁Δ𝜔+𝑎𝑏𝜔−𝜔𝜃_𝑐1+𝑘𝜃_𝑐𝜔=0, (3.12) which leads to ∫_Ω(1/𝜇₁)(𝑎−𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))𝜔²=0, a contradiction. This proves the assertion, so it is verified that the multiplicity of 𝜇₁ is one and index(𝑇(𝑎;⋅),0)=−1 for 𝑎<𝑎<𝑎₂(1). According to global bifurcation theory [12], there exists a continuum 𝐶₀ of zeros of 𝐺(𝑎;𝜔,𝜒)=0 in 𝑅⁺×𝑋 bifurcating from (𝑎;0,0), and all zeros of 𝐺(𝑎;𝜔,𝜒) close to (𝑎;0,0) lie on the curve whose existence was proved by Lemma 3.1. Let 𝐶₁ be the maximal continuum defined by 𝐶₁=𝐶₀−{(𝑎(𝑠);𝑠(Φ+𝜙(𝑠)),𝑠(Ψ+𝜑(𝑠)))∶−𝛿<𝑠<0}. Then, 𝐶₁ consists of the curve {(𝑎(𝑠);𝑠(Φ+𝜙(𝑠)),𝑠(Ψ+𝜑(𝑠)))∶−𝛿<𝑠<0} in the neighborhood of the bifurcation point (𝑎;0,0). Let 𝐶={(𝑎;𝑢,𝑣)∶𝑈=𝜔,𝑉=𝜃_𝑐+𝜒,(𝜔,𝜒)∈𝐶₁}. Then 𝐶 is the solution branch of (1.2) which bifurcates from (𝑎;0,𝜃_𝑐) and remains positive in a small neighborhood of (𝑎;0,𝜃_𝑐) and 𝐶⊂𝑃. Thus the continuum 𝐶−{(𝑎;0,𝜃_𝑐)} must satisfy one of the following three alternatives.(i)𝐶 contains in its closure points (𝑎;0,𝜃_𝑐) and (𝑎;0,𝜃_𝑐), where 𝐼−𝑇^′(𝑎) is not invertible, and 𝑎≠𝑎.(ii)𝐶 is joining up from (𝑎;0,𝜃_𝑐) to ∞ in 𝑅×𝑋.(iii)𝐶 is containing points of the form (𝑎;𝑢,𝜃_𝑐+𝑣) and (𝑎;−𝑢,𝜃_𝑐−𝑣), where (𝑢,𝑣)≠(0,0).
Next, we prove that 𝐶−{(𝑎;0,𝜃_𝑐)}⊂𝑃. Assume that 𝐶−{(𝑎;0,𝜃_𝑐)}⊆𝑃̸. Then there exists (𝑎;𝑢,𝑣)∈(𝐶−{(𝑎;0,𝜃_𝑐)})∩𝜕𝑃 and sequence {(𝑎_𝑛;𝑢_𝑛,𝑣_𝑛)}⊂𝐶∩𝑃,𝑢_𝑛>0,𝑣_𝑛>0 such that (𝑎_𝑛;𝑢_𝑛,𝑣_𝑛)→(𝑎;𝑢,𝑣) when 𝑛→∞. It is easy to get that 𝑢∈𝜕𝑃₁ or 𝑣∈𝜕𝑃₁. Suppose 𝑢∈𝜕𝑃₁, then 𝑢≥0,𝑥∈Ω. Hence, we find either 𝑥₀∈Ω such that 𝑢(𝑥₀)=0 or 𝑥₀∈𝜕Ω such that 𝜕𝑢/𝜕𝑛∣_𝑥0=0. Since 𝑢 satisfies 𝑏𝑣−Δ𝑢=𝑎−𝑢−(𝑢𝑣)𝑢|1+𝑚)(1+𝑘𝑢,_𝜕Ω=0. (3.13) It follows from the maximum principle that 𝑢≡0. Similarly, we can show that 𝑣≡0 for 𝑣∈𝜕𝑃₁.
Thus we only need the following three cases: (i)(𝑢,𝑣)≡𝜃𝑎,0,(ii)(𝑢,𝑣)≡0,𝜃_𝑐,(iii)(𝑢,𝑣)≡(0,0). (3.14)
Suppose that (𝑢,𝑣)≡(𝜃𝑎,0). Then (𝑎_𝑛;𝑢_𝑛,𝑣_𝑛)→(𝑎;𝜃𝑎,0) when 𝑛→∞. Let 𝑉_𝑛=𝑣_𝑛/||𝑣_𝑛||_∞; then 𝑉_𝑛 satisfies −Δ𝑉_𝑛=𝑐−𝑣_𝑛+𝑑𝑢_𝑛1+𝑚𝑢_𝑛1+𝑘𝑣_𝑛𝑉_𝑛,𝑉_𝑛𝜕Ω=0. (3.15) Thanks to 𝐿^𝑃 estimates and Sobolev embedding theorem, there exists a convergent subsequence of 𝑉_𝑛, which we still denote by 𝑉_𝑛, such that 𝑉_𝑛→𝑉 in 𝐶¹₀(Ω) as 𝑛→∞, and 𝑉≥0,≢0,𝑥∈Ω because of ‖𝑉‖=1. So taking the limit in (3.15) as 𝑛→∞, we get−Δ𝑉=𝑐+𝑑𝜃𝑎𝑎|1+𝑚𝜃𝑉,𝑉_𝜕Ω=0. (3.16) It follows from the maximum principle that 𝑉>0,𝑥∈Ω, which implies 𝑐=𝜆₁(−𝑑𝜃𝑎/(1+𝑚𝜃𝑎)). This contradicts 𝑐>𝜆₁.
Suppose that (𝑢,𝑣)≡(0,𝜃_𝑐). Then (𝑎_𝑛;𝑢_𝑛,𝑣_𝑛)→(𝑎;0,𝜃_𝑐) as 𝑛→∞. Let 𝑈_𝑛=𝑢_𝑛/||𝑢_𝑛||_∞; then 𝑈_𝑛 satisfies −Δ𝑈_𝑛=𝑎_𝑛−𝑢_𝑛−𝑏𝑣_𝑛1+𝑚𝑢_𝑛1+𝑘𝑣_𝑛𝑈_𝑛,𝑈_𝑛𝜕Ω=0. (3.17) Similarly, By 𝐿^𝑃 estimates and and Sobolev embedding theorem, there exists a convergent subsequence of 𝑈_𝑛, which we still denote by 𝑈_𝑛, such that 𝑈_𝑛→𝑈 in 𝐶¹₀(Ω) as 𝑛→∞, and 𝑈≥0,≢0,𝑥∈Ω because of ‖𝑉‖=1. So taking limit in (3.17) as 𝑛→∞, we obtain−Δ𝑈=𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝑈,𝑈|_𝜕Ω=0. (3.18) It follows from the maximum principle that 𝑈>0,𝑥∈Ω. Hence 𝑎=𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). A contradiction with 𝑎≠𝑎.
Suppose that (𝑢,𝑣)≡(0,0). Similar to the previously mentioned, we can get contradiction.
Thus 𝐶−{(𝑎;0,𝜃_𝑐)}⊂𝑃. By Lemma 2.1, we have 0≤𝑈≤𝑎,𝜃_𝑐≤𝑉≤𝑐+𝑑𝑎/(1+𝛼𝑎). Thanks to 𝐿^𝑃 estimates and and Sobolev embedding theorem, then there exists a constant 𝑀>0 such that ||𝑈||_𝐶¹,||𝑉||_𝐶¹≤𝑀. Hence the global bifurcation 𝐶 of positive solutions of (1.2) bifurcating at (𝑎;0,𝜃_𝑐) contains points with 𝑎 is arbitrarily large in 𝑃.

In the following, we will study the stability of the bifurcation solution. Let 𝑋₁=[𝐶^2,𝛼(Ω)×𝐶^2,𝛼(Ω)∩𝑋],𝑌=[𝐶^𝛼(Ω)×𝐶^𝛼(Ω)], where 0<𝛼<1.𝑖∶𝑋₁→𝑌 is the inclusion mapping. Since 𝐿₁ is the linearized operator at (𝑎;0,𝜃_𝑐) for (1.2). By the proof of Lemma 3.1, we have 𝑁(𝐿₁)= span{(Φ,Ψ)}, Codim𝑅(𝐿₁)=1, and 𝑅(𝐿₀)={(𝑢,𝑣)∈𝑋∶∫_Ω𝑢Φ𝑑𝑥=0}. Since 𝑖(Φ,Ψ)∈𝑅(𝐿₁), so it follows from [21] that 0 is an 𝑖-simple eigenvalue of 𝐿₁.

Lemma 3.4. 0 is the eigenvalue of 𝐿₁ with the largest real part, and all the other eigenvalue of 𝐿₁ lie in the left half complex plane.

Proof. We assume that 𝜇₀ is the eigenvalue of 𝐿₁ with the largest real part Re𝜇₀>0, and (𝜉,𝜂) is the corresponding eigenfunction; then 𝐿₁(𝜉,𝜂)=𝜇₀(𝜉,𝜂), equivalently, Δ𝜉+𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜉=𝜇₀𝜉,𝑥∈Ω,Δ𝜂+𝑐−2𝜃_𝑐𝜂+𝑑𝜃_𝑐1+𝑘𝜃_𝑐𝜉=𝜇₀𝜂,𝑥∈Ω,𝜂=𝜉=0,𝑥∈𝜕Ω. (3.19)
Suppose that 𝜉≡0. Then 𝜇₀ is an eigenvalue of the operator (Δ+(𝑐−2𝜃_𝑐)𝐼), and then 𝜇₀∈𝑅 and 𝜇₀<0, a contradiction. Thus Φ≡0. It follows that 𝜇₀ is an eigenvalue of the operator (Δ+(𝑎−𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))𝐼). Since 𝑎=𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)),0 is the principal eigenvalue of the operator (Δ+(𝑎−𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))𝐼); furthermore, 𝜇₀≤0. This contradicts the assumption, so assumption does not hold. Which proves our conclusion.

We will use the linearized stability theory from [22]. Let 𝐿(𝑢(𝑠),𝑣(𝑠),𝑎(𝑠)), and 𝐿(𝑎;0,𝜃_𝑐) be the linearized operators of (1.2) at (𝑢(𝑠),𝑣(𝑠),𝑎(𝑠)), and (𝑎;0,𝜃_𝑐), respectively. It follows that from Lemma 7, Corollary 1.13, and Theorem 1.16 [22, 23] that Lemma 3.5 holds.

Lemma 3.5. There exists 𝐶¹-function: 𝑎→(𝑀(𝑎),𝛾(𝑎)), and 𝑠→(𝑁(𝑠),𝜋(𝑠)), defined from the neighborhood of 𝑎 and 0 into 𝑋₁×𝑅, respectively, such that 𝛾(𝑎)=𝜋(0)=0,𝑀(𝑎)=𝑁(0)=(Φ,Ψ) and 𝐿𝑎;0,𝜃_𝑐𝑀(𝑎)(𝑎)𝑀(𝑎)|𝑎|𝐿(𝑢(𝑠)(𝑠)(𝑠𝑁(𝑠)(𝑠)𝑁(𝑠)|𝑠|=𝛾,𝑓𝑜𝑟𝑎−≪1,,𝑣,𝑎))=𝜋,𝑓𝑜𝑟≪1, (3.20) where 𝑀(𝑎)=(𝜙₁(𝑎),𝜙₂(𝑎)),𝑁(𝑠)=(𝜑₁(𝑎)𝜑₂(𝑎)). Moreover 𝛾^′(𝑎)≠0, whence 𝜋(𝑠)≠0,𝜋(𝑠) and −𝑠𝑎^′(𝑠)𝛾^′(𝑎) have the same sign for |𝑠|≪1. Where 𝛾^′(𝑎) is the derivative of 𝛾(𝑎) with respect to 𝑎 at 𝑎=𝑎, and 𝑎^′(𝑠) is the derivative of 𝑎(𝑠) with respect to 𝑠.

Lemma 3.6. The derivative of 𝛾(𝑠) with respect to 𝑎 at 𝑎 is positive.

Proof. It follows from 𝐿(𝑎;0,𝜃_𝑐)𝑀(𝑎)=𝛾(𝑎)𝑀(𝑎),for,|𝑎−𝑎|≪1, that is, Δ𝜙₁+𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜙₁=𝛾(𝑎)𝜙₁,𝑥∈Ω,Δ𝜙₂+𝑐−2𝜃_𝑐𝜙₂+𝑑𝜃_𝑐1+𝑘𝜃_𝑐𝜙₁=𝛾(𝑎)𝜙₂𝜙,𝑥∈Ω,₁=𝜙₂=0,𝑥∈𝜕Ω. (3.21) Since |𝑎−𝑎|≪1, so |𝛾(𝑎)|≪1. Clearly, 𝜙₁≢0, otherwise, 𝜙₁≡0, then 𝜙₂≡0, a contradiction. Hence 𝛾(𝑎) is an eigenvalue of the operator (Δ+(𝑎−𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))𝐼). We consider Φ>0, then 𝜙₁=𝜙₁(𝑎)>0, as |𝑎−𝑎|≪1. So 𝛾(𝑎) is the principal eigenvalue of the operator (Δ+(𝑎−𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))𝐼), and 𝛾(𝑎) is increasing with respect to 𝑎 as |𝑎−𝑎|≪1. Moreover, 𝛾^′(𝑎)≠0. Hence 𝛾^′(𝑎)>0.

Lemma 3.7. The derivative of 𝑎(𝑠) with respect to 𝑠 at 𝑠=0 satisfies 𝑎^′(0)_ΩΦ²𝑑𝑥=_Ω1−𝑏𝑚𝜃_𝑐1+𝑘𝜃_𝑐Φ³𝑑𝑥+_Ω𝑏Ψ1+𝑘𝜃_𝑐²Φ²𝑑𝑥. (3.22)

Proof. By substituting (𝑢(𝑠),𝑣(𝑠),𝑎(𝑠)) into (1.2), differentiating with respect to 𝑠, and then setting 𝑠=0, we find that −Δ𝜙^′(0)=𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜙^′(0)+𝑎^′(0)−Φ−𝑏Ψ−𝑚𝜃_𝑐Φ1+𝑘𝜃_𝑐1+𝑘𝜃_𝑐²Φ, (3.23) where 𝜙^′(0) is the derivative of 𝜙 with respect to 𝑠 at 𝑠=0.
Taking the inner product with Φ, using Green's formula, and noting the definition of Φ, we have 𝑎^′(0)_ΩΦ²𝑑𝑥=_Ω1−𝑏𝑚𝜃_𝑐1+𝑘𝜃_𝑐Φ³𝑑𝑥+_Ω𝑏Ψ1+𝑘𝜃_𝑐²Φ²𝑑𝑥. (3.24)

It follows from Lemmas 3.4–3.7 that we obtain the following Theorem.

Theorem 3.8. Let 𝜎=∫_Ω(1−𝑏𝑚𝜃_𝑐/(1+𝑘𝜃_𝑐))Φ³𝑑𝑥+∫_Ω(𝑏Ψ/(1+𝑘𝜃_𝑐)²)Φ²𝑑𝑥. If 𝜎>0, then bifurcation solution (𝑢(𝑠),𝑣(𝑠)) is stable; If 𝜎<0, then bifurcation solution (𝑢(𝑠),𝑣(𝑠)) is unstable.

In Section 2, from Theorems 2.5–2.6, we can obtain the sufficient condition and necessary condition on the existence of positive solution and find that there exists a gap between 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) and 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑚𝜃_𝑎)(1+𝑘𝜃_𝑐)) when 𝑐>𝜆₁. Next, we will consider the multiplicity, stability, and uniqueness of positive solutions in the gap.

Theorem 3.9. Assume 𝑐>𝜆₁ and ∫_Ω(1−𝑏𝑚𝜃_𝑐/(1+𝑘𝜃_𝑐))Φ³𝑑𝑥<0. Then there exist 𝜀>0 such that bifurcation positive solution (𝑢(𝑠),𝑣(𝑠)) is nondegenerate and unstable for 𝑎∈(𝑎−𝜀,𝑎) and 𝑑≪1. Moreover, Problem (1.2) has at least two positive solutions.

Proof. We first prove that bifurcation positive solution (𝑢(𝑠),𝑣(𝑠)) is nondegenerate and unstable. To this end, it suffices to show that there exists a sufficiently small 𝜀>0 such that for 𝑎∈(𝑎−𝜀,𝑎), any positive solution (𝑢(𝑠),𝑣(𝑠)) of (1.2) is nondegenerate and the linearized eigenvalue problem: 𝑎(𝑠)(𝑠)−(𝑠)−Δ𝜉−−𝑢𝑏𝑣(1+𝑚𝑢(𝑠))²((𝑠(𝑠)1+𝑘𝑣))𝜉+𝑏𝑢(1+𝑚𝑢(𝑠))(1+𝑘𝑣(𝑠))²(𝑠)+(𝑠)𝜂=𝜇𝜉,𝑥∈Ω,−Δ𝜂−𝑐−2𝑣𝑑𝑢(1+𝑚𝑢(𝑠))(1+𝑘𝑣(𝑠))²(𝑠)𝜂−𝑑𝑣(1+𝑚𝑢(𝑠))²(1+𝑘𝑣(𝑠))=𝜇𝜂,𝑥∈Ω,𝜉=𝜂=0,𝑥∈𝜕Ω, (3.25) has a unique eigenvalue 𝜇_∗ and Re(𝜇_∗)<0 with algebra multiplicity one.
Let {𝜀_𝑖>0} and {𝑑_𝑖>0} be sequences which approach 0 as 𝑖→∞. Due to 𝑎=𝑎+𝑎^′(0)𝑠+𝑂(𝑠²), we can set sequences {𝜀_𝑖>0} and {𝑎_𝑖} such that 𝑎_𝑖∈(𝑎−𝜀_𝑖,𝑎) and 𝑠_𝑖→0 as 𝑖→∞. It follows that (𝑢_𝑖,𝑣_𝑖) is a solution of (1.2). Then the corresponding linearized problem (3.25) can become the following form: 𝐿_𝑖𝜉_𝑖𝜂_𝑖=𝜇_𝑖𝜉_𝑖𝜂_𝑖,𝐿_𝑖=𝑀_𝑖¹¹𝑀_𝑖¹²𝑀_𝑖²¹𝑀_𝑖²², (3.26) where (𝜉_𝑖,𝜂_𝑖)≢(0,0) and 𝑀_𝑖¹¹=−Δ−𝑎_𝑖−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘𝑣_𝑖,𝑀_𝑖¹²=𝑏𝑢_𝑖1+𝑚𝑢_𝑖1+𝑘𝑣_𝑖²,𝑀_𝑖²¹=−𝑑𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘𝑣_𝑖,𝑀_𝑖²²=−Δ−𝑐−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚𝑢_𝑖1+𝑘𝑣_𝑖². (3.27) Observe that as 𝑖→∞,  𝐿_𝑖^𝜉𝑖_𝜂𝑖 converges to𝐿₀𝜉𝜂=−Δ𝜉−𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝜉00−Δ𝜂−𝑐−2𝜃_𝑐𝜂. (3.28) It is easy to get that 0 is a simple eigenvalue of the operator 𝐿₀ with corresponding eigenfunction (𝜉,𝜂)^𝑇=(Φ,0)^𝑇. Moreover, all the other eigenvalues of 𝐿₀ are positive and stand apart from 0. Therefore, using perturbation theory [24], we get that for large 𝑖,𝐿_𝑖 has a unique eigenvalue 𝜇_𝑖 which is close to zero. In addition, all the other eigenvalues of 𝐿_𝑖 have positive real parts and stand apart from 0. Note that 𝜇_𝑖 is simple real eigenvalue which converges to zero, and we can take the corresponding eigenfunction (𝜉_𝑖,𝜂_𝑖)^𝑇 such that (𝜉_𝑖,𝜂_𝑖)→(Φ,0) as 𝑖→∞. If we show that Re𝜇_𝑖<0 for large 𝑖, then the result follows. By multiplying Φ to the first equation of 𝐿_𝑖(𝜉_𝑖,𝜂_𝑖)^𝑇=𝜇_𝑖(𝜉_𝑖,𝜂_𝑖)^𝑇 and integrating over Ω, we obtain −_ΩΦΔ𝜉_𝑖−_Ω𝑎_𝑖−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘𝑣_𝑖Φ𝜉_𝑖+_Ω𝑏𝑢_𝑖Φ𝜂_𝑖1+𝑚𝑢_𝑖1+𝑘𝑣_𝑖²=_Ω𝜇_𝑖Φ𝜉_𝑖. (3.29) Multiplying the first equation of (1.2) with (𝑎,𝑢,𝑣)=(𝑎_𝑖,𝑢_𝑖,𝑣_𝑖) by 𝜉_𝑖 and integrating, we have −_Ω𝜉_𝑖Δ𝑢_𝑖−_Ω𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑚𝑢_𝑖1+𝑘𝑣_𝑖𝑢_𝑖𝜉_𝑖. (3.30) Due to 𝑢_𝑖=𝑠_𝑖Φ+𝑂(𝑠²_𝑖), the previous equation becomes −_ΩΦΔ𝜉_𝑖−_Ω𝜉_𝑖Φ𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑚𝑢_𝑖1+𝑘𝑣_𝑖+𝑂𝑠²_𝑖. (3.31) Using (3.29) and (3.31), we have _Ω1−𝑏𝑚𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘𝑣_𝑖𝑢_𝑖Φ𝜉_𝑖+_Ω𝑏𝑢_𝑖Φ𝜂_𝑖1+𝑚𝑢_𝑖1+𝑘𝑣_𝑖²=_Ω𝑢_𝑖Φ𝜉_𝑖. (3.32) Recall that (𝑢_𝑖,𝑣_𝑖)=(𝑠_𝑖Φ+𝑂(𝑠²_𝑖),𝜃_𝑐+𝑠_𝑖Ψ_𝑑𝑖+𝑂(𝑠²_𝑖)), here Ψ_𝑑𝑖 is Ψ defined in Remark 3.2, and so dividing the previous equation by 𝑠_𝑖 and taking the limit, we have lim_𝑖→∞𝜇_𝑖𝑠_𝑖=∫_Ω1−𝑏𝑚𝜃_𝑐/1+𝑘𝜃_𝑐Φ³∫_ΩΦ²,<0 (3.33) which implies that Re𝜇_𝑖<0 for large 𝑖. This proved our claim.
Next, we apply the method in [25] to show the remaining part of Theorem 3.9. A contradiction argument will be used; we assume that (1.2) has a unique positive solution (𝑢,𝑣), then this solution must be bifurcated from (0,𝜃_𝑐). Since there exists a positive solution near 𝑎 by the local bifurcation theory. So (𝑢,𝑣) is nondegenerate, and the corresponding linearized eigenvalue problem has a unique eigenvalue 𝜇 with algebra multiplicity one such that Re𝜇<0. Due to these facts, it is easy to show that 𝐼−𝐹^′(𝑢,𝑣) is invertible and does not have property 𝛼 on 𝑊₍𝑢_,𝑣₎; it follows that index(𝐹,(𝑢,𝑣))=(−1)¹=−1 by Lemma 2.2 (ii). Finally, Using Lemmas 2.3–2.4 and the additivity property of the index, we obtain 1=index_𝑊(𝐹,𝐷)=index_𝑊(𝐹,(0,0))+index_𝑊𝐹,𝜃_𝑎,0+index_𝑊𝐹,0,𝜃_𝑐+index_𝑊(𝐹,(𝑢,𝑣))=0+0+1−1=0, (3.34) which derives a contradiction. Hence the Proof is complete.

Remark 3.10. In Theorem 3.9, the multiplicity can be shown easily when 𝑚≥0. Note that 𝑎^′(0)<0 for a sufficiently small 𝑑, since ∫_Ω(1−𝑏𝑚𝜃_𝑐/(1+𝑘𝜃_𝑐))Φ³<0, and so 𝑎=𝑎(𝑠)∈(𝜆₁,𝑎). Since there is no positive solution of (1.2) if 𝑎≤𝜆₁(𝑏𝜃_𝑐/(1+𝑚𝜃_𝑎)(1+𝑘𝜃_𝑐)) by Theorem 2.5 (i) when 𝑚≥0 and 𝑐>𝜆₁. Therefore we easily see that there must be at least two positive solutions for 𝑎∈(𝑎^∗,𝑎) and some 𝑎^∗∈(𝜆₁(𝑏𝜃_𝑐/(1+𝑚𝜃_𝑎)(1+𝑘𝜃_𝑐)),𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))).

4. Multiplicity, Uniqueness and Stability as 𝑘 or 𝑚 Is Large

In this section, taking 𝑘 or 𝑚 as a parameter, we investigate the multiplicity, stability, and uniqueness of positive solutions of (1.2) as 𝑘 or 𝑚 is large. In the following, we will always assume that 𝑐>𝜆₁, and let 𝑏,𝑐,𝑑 be fixed, unless otherwise specified.

Firstly, we consider the case that 𝑘 is large and 𝑎 is bounded away from 𝜆₁. Hence the upper solution 𝜃_𝑎 for 𝑢 and the lower solution 𝜃_𝑐 for 𝑣 do not depend on 𝑘 when 𝑎>𝜆₁ and 𝑐>𝜆₁.

Lemma 4.1. For any small 𝜀>0, there exists 𝐾(𝜀) such that for 𝑘≥𝐾(𝜀), (1.2) has at least one positive solution (𝑢,𝑣) which satisfies 𝜃_{(𝑎−𝜀)}≤𝑢≤𝜃_𝑎,𝜃_𝑐≤𝑣≤𝜃_(𝑐+𝜀). (4.1)

Proof. Since the proof of Lemma 4.1 is similar to the proof of Lemma 3 in [19], we omit it.

Lemma 4.2. (𝑖) If 𝑘→∞, then any positive solution of (1.2) approaches (𝜃_𝑎,𝜃_𝑐).
(𝑖𝑖) There exists a large 𝐾(𝜖) such that any positive solution of (1.2) is nondegenerate and linearly stable when 𝑘>𝐾(𝜖).

Proof. (i) Let 𝑘→∞, we show that the compact operator 𝐹(𝑢,𝑣) converges to 𝐹(𝑢,𝑣), where 𝐹(𝑢,𝑣)=(−Δ+𝑃)⁻¹𝑢()𝑣()𝑎−𝑢+𝑃𝑢𝑐−𝑣+𝑃𝑣. (4.2) It follows that any positive solution of (1.2) converges to the fixed point of 𝐹(𝑢,𝑣) in this case. It is easy to see that (𝜃_𝑎,𝜃_𝑐) is a unique fixed point of 𝐹(𝑢,𝑣), so the positive solutions of (1.2) could not converge to semitrivial solutions when 𝑘→∞. Thus the conclusion is complete.
(ii) We use a contradiction method. Assume that there exists 𝑘_𝑖→∞, 𝜇_𝑖 with Re𝜇_𝑖≤0 and (𝜉_𝑖,𝜂_𝑖)≢(0,0) with ‖𝜉_𝑖‖²_𝐿2+‖𝜂_𝑖‖²_𝐿2=1 such that −Δ𝜉_𝑖−𝑎−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘_𝑖𝑣_𝑖𝜉_𝑖+𝑏𝑢_𝑖1+𝑚𝑢_𝑖1+𝑘_𝑖𝑣_𝑖²𝜂_𝑖=𝜇_𝑖𝜉_𝑖,𝑥∈Ω,−Δ𝜂_𝑖−𝑐−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚𝑢_𝑖1+𝑘_𝑖𝑣_𝑖²𝜂_𝑖−𝑑𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘_𝑖𝑣_𝑖𝜉_𝑖=𝜇_𝑖𝜂_𝑖𝜉,𝑥∈Ω,_𝑖=𝜂_𝑖=0,𝑥∈𝜕Ω, (4.3) where (𝑢_𝑖,𝑣_𝑖) is a positive solution of (1.2) with 𝑘=𝑘_𝑖. By computing, we have 𝜇_𝑖=_Ω∇𝜉_𝑖²−_Ω𝑎−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚𝑢_𝑖²1+𝑘_𝑖𝑣_𝑖𝜉_𝑖²+_Ω𝑏𝑢_𝑖𝜂_𝑖𝜉_𝑖1+𝑚𝑢_𝑖1+𝑘_𝑖𝑣_𝑖²+_Ω∇𝜂_𝑖²−_Ω𝑑𝑣_𝑖𝜉_𝑖𝜂_𝑖1+𝑚𝑢_𝑖²1+𝑘_𝑖𝑣_𝑖−_Ω𝑐−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚𝑢_𝑖1+𝑘_𝑖𝑣_𝑖²𝜂_𝑖², (4.4) where 𝜉_𝑖,𝜂_𝑖 are the complex conjugates of 𝜉_𝑖,𝜂_𝑖. From Lemma 2.1, we know that 0≤𝑢_𝑛≤𝜃_𝑎 and 𝜃_𝑐≤𝑣_𝑖≤𝑐+𝑑𝑎/(1+𝑚𝑎). It follows that Im𝜇_𝑖 is bounded and Re𝜇_𝑖 is bounded from the following. Thus 𝜇_𝑖 is bounded as we assume Re𝜇_𝑖≤0. Hence we can suppose that 𝜇_𝑖→𝜇 and Re𝜇≤0. Thanks to 𝐿^𝑝 estimate, we obtain ‖𝜉_𝑖‖_𝑊2,2 and ‖𝜂_𝑖‖_𝑊2,2 are bounded. Hence we may assume that 𝜉_𝑖→𝜉 and 𝜂_𝑖→𝜂 in 𝐻¹₀ strongly, here (𝜉,𝜂)≢(0,0). Setting 𝑖→∞ in (3.1) and (3.2), we know that 𝜉,𝜂 satisfy the following two single equations weakly (then strongly):−Δ𝜉−𝑎−2𝜃_𝑎𝜉=𝜇𝜉,𝑥∈Ω,−Δ𝜁−𝑐−2𝜃_𝑐𝜂=𝜇𝜂,𝑥∈Ω,𝜉=𝜂=0,𝑥∈𝜕Ω. (4.5) Clearly 𝜇∈𝑅. If 𝜉≢0, then 𝜇=𝜆₁(−𝑎+2𝜃_𝑎)>𝜆₁(−𝑎+𝜃_𝑎)=0. However, Re𝜇≤0. Hence 𝜉≡0. Similarly, we have 𝜂≡0, a contradiction; hence the proof is complete.

Theorem 4.3. (𝑖) Assume that 𝜆₁<𝑎<𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). Then (1.2) has no positive solution when 𝑘 is sufficiently large.
(𝑖𝑖) Assume that 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) and 𝑘 is sufficiently large. Then (1.2) has a unique positive solution, and it is asymptotically stable.

Proof. (i) Assume that there exists a positive solution (𝑢_∗,𝑣_∗) of (1.2) for sufficiently large 𝑘. It is easy to show that index_𝑊(𝐹,(𝑢_∗,𝑣_∗))=1 by Lemma 4.2. By Lemmas 2.3–2.4 and the additivity property of the index, we obtain 1=index_𝑊𝐹,𝐷^′=index_𝑊(𝐹,(0,0))+index_𝑊𝐹,𝜃_𝑎,0+index_𝑊𝐹,0,𝜃_𝑐+index_𝑊𝐹,𝑢_∗,𝑣_∗=0+0+1+1=2, (4.6) which gives a contradiction.
(ii) From Theorem 2.6, the existence is trivial. Since 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) and 𝑐>𝜆₁, for 𝑘 sufficiently large, the nondegenerate positive solutions may not converge to semitrivial solutions by the proof of Lemma 4.2 (i). We need only to show the uniqueness. It follows from compactness and nondegeneracy that 𝐹 has at most finitely many positive fixed points in the region 𝐷^′. We denote them by (𝑢_𝑖,𝑣_𝑖) for 𝑖=1,2,…,𝑙. From the proof of (i), we obtain index_𝑊(𝐹,(𝑢_𝑖,𝑣_𝑖))=1. Applying Lemmas 2.3–2.4 and the additivity property of the index again, we have 1=index_𝑊𝐹,𝐷^′=index_𝑊(𝐹,(0,0))+index_𝑊𝐹,𝜃_𝑎,0+index_𝑊𝐹,0,𝜃_𝑐+^𝑙_𝑖=1index_𝑊𝐹,𝑢_𝑖,𝑣_𝑖=0+0+0+𝑙=𝑙. (4.7) Hence the uniqueness is obtained. The stability has been given in Lemma 4.2.

In the following, we investigate the case when 𝑚 is large. This part is motivated by the work of Du and Lou in [13, 14, 18], and many of our methods used nextly come from their work.

Theorem 4.4. For any 𝜖>0 to be small, there that exists 𝑀=𝑀(𝜖) is large such that for 𝑚≥𝑀,(i) if 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), then (1.2) has at least two positive solutions;(ii) if 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), then (1.2) has a unique positive solution and it is asymptotically stable.

Theorem 4.4 is the main result we will prove that when 𝑚 is large. The cases 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))) and 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) will be treated separately. First, we deal with the multiplicity in Theorem 4.4. Similar to the method of [19], if 𝑎∈[𝜆₁+𝜖,∞), and 𝑚 is sufficiently large, then we can get the following result on (1.2).

Lemma 4.5. For any given small 𝜖, there exists that 𝑀=𝑀(𝜖) such that if 𝑎≥𝜆₁+𝜖 and 𝑚≥𝑀, (1.2) has a positive solution (𝑢,𝑣), which satisfy 𝜃_{(𝑎−𝜖/2)}≤𝑢≤𝜃_𝑎,𝜃_𝑐≤𝑣≤𝜃_{(𝑐+𝜖/2)}. (4.8)

Lemma 4.6. For any 𝜖>0 to be small and any 𝐴>𝜆₁, there exists that 𝑀=𝑀(𝜖,𝐴)>0 is large such that if 𝑎∈(𝜆₁+𝜖,𝐴] and 𝑚≥𝑀, then any positive solution which satisfies (4.8) is nondegenerate and linearly stable.

Proof. Assume that 𝑎∈(𝜆₁+𝜖,𝐴] and 𝑚 is large, then we can easily get that (1.2) is a regular perturbation of ()()−Δ𝑢−𝑎−𝑢𝑢=0,𝑥∈Ω,−Δ𝑣−𝑐−𝑣𝑣=0,𝑥∈Ω,𝑢=𝑣=0,𝑥∈𝜕Ω. (4.9) It is well know that (4.9) has a unique positive solution (𝜃_𝑎,𝜃_𝑐) which is linearly stable. So the positive solutions cannot bifurcate from semitrivial ones. Hence, Lemma 4.6 is proved by a standard regular perturbation argument. We omit the details.

Proof of (i) of Theorem 4.4. For any 𝜖>0 to be small, let 𝑀=max𝑀(𝜖),𝑀𝜖,𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐, (4.10) where 𝑀(𝜖) and 𝑀(𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))) are defined in Lemmas 4.5 and 4.6, respectively. Assume that for some 𝑚≥𝑀 and some 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), Thus the unique positive solution (𝑢,𝑣) must be the one found in Lemma 4.5. It follows from Lemma 4.5 that 𝐼−𝐹^′(𝑢,𝑣) is invertible in 𝑋 and 𝐹^′(𝑢,𝑣) has no eigenvalue greater than one. Hence index_𝑊(𝐹,(𝑢,𝑣))=(−1)⁰=1. Applying Lemmas 2.3–2.4 and the additivity property of the index, we obtain 1=index_𝑊𝐹,𝐷^′=index_𝑊(𝐹,(0,0))+index_𝑊𝐹,𝜃_𝑎,0+index_𝑊𝐹,0,𝜃_𝑐+index_𝑊(𝐹,(𝑢,𝑣))=0+0+1+1=2, (4.11) a contradiction. Hence the proof if complete.

Part (i) in Theorem 4.4 implies that when 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))) and 𝑚 is large, (1.2) has at least two positive solutions. In the following, we will show that (1.2) has only two types of positive solutions in this case, one of which is close to (𝜃_𝑎,𝜃_𝑐) and asymptotically stable and the other is close to (0,𝜃_𝑐) which is not stable.

Theorem 4.7. For any 𝜖,𝛿>0 to be small, there exists that 𝑀=𝑀(𝜖,𝛿)>0 is large such that if 𝑚≥𝑀 and 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), one obtains either (𝑖)‖𝑢−𝜃_𝑎‖_𝐶¹+‖𝑣−𝜃_𝑐‖_𝐶¹≤𝛿 or (𝑖𝑖)‖𝑢‖_𝐶¹+‖𝑣−𝜃_𝑐‖_𝐶¹≤𝛿, where (𝑢,𝑣) is any positive solution of (1.2). In particular, if (𝑖𝑖) occurs, by choosing 𝑀(𝜖,𝛿) suitably larger, one gets ‖𝑚𝑢−𝑤‖_𝐶¹≤𝛿, where 𝑤 is a positive solution of the following equation: −Δ𝑤−𝑎−𝑏𝜃_𝑐(1+𝑤)1+𝑘𝜃_𝑐𝑤=0,𝑤|_𝜕Ω=0. (4.12)

Proof. Suppose that the conclusion does not hold. Then there exist 𝑚_𝑖→∞𝑎_𝑖∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))) and a positive solution (𝑢_𝑖,𝑣_𝑖) of (1.2) with (𝑎,𝑚)=(𝑎_𝑖,𝑚_𝑖) such that (𝑢_𝑖,𝑣_𝑖) is bounded away from (𝜃_𝑎,𝜃_𝑐) and (0,𝜃_𝑐). We may assume that 𝑎_𝑖→𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))] and (1/(1+𝑚_𝑖𝑢_𝑖)(1+𝑘𝑣_𝑖))⇀^𝐿2ℎ with 0<ℎ<1. Since 𝜃_𝑐≤𝑣_𝑖≤𝜃_{(𝑐+𝑑/𝑚𝑖)}, we have 𝑣−−→_𝑖^𝐶1𝜃_𝑐 as 𝑖→∞. Thanks to 𝐿^𝑝 estimate and Sobolev embedding theorems, we may assume that 𝑢−−→_𝑖^𝐶1𝑢 and 𝑢 satisfies Δ𝑢+𝑎−𝑢−𝑏𝜃_𝑐ℎ𝑢=0,𝑢≥0,𝑢|_𝜕Ω=0. (4.13) If 𝑢≡0, then (𝑢_𝑖,𝑣_𝑖)−−→^𝐶1(𝑢,𝜃_𝑐)≡(0,𝜃_𝑐), which contradicts our assumption that (𝑢_𝑖,𝑣_𝑖) is bounded away from (0,𝜃_𝑐). If 𝑢≥0,≢0, from maximum principle, we have 𝑢>0 in Ω. Hence ℎ≡0 and 𝑢≡𝜃_𝑎, which also contradicts our assumption. Thus, the first part of the proof is complete.
To complete the proof, it suffices to prove that if 𝑚_𝑖→∞,𝑎_𝑖∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))) and ‖𝑢_𝑖‖_𝐶¹+‖𝑣_𝑖−𝜃_𝑐‖_𝐶¹→0, then 𝑚_𝑖𝑢_𝑖 approaches some positive solution of (4.12) with 𝑎=𝑎_𝑖 in the 𝐶¹ norm. It is easy to see that (4.12) has a positive solution when 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))). First we claim that 𝑚_𝑖‖𝑢_𝑖‖_∞ is uniformly bounded. If this is not true, we may assume that 𝑚_𝑖‖𝑢_𝑖‖_∞→∞. Let 𝑢_𝑖=𝑢_𝑖/‖𝑢_𝑖‖_∞. Then we have Δ𝑢_𝑖+𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖𝑢_𝑖=0,𝑢_𝑖∞=1,𝑢_𝑖𝜕Ω=0. (4.14) Thanks to standard elliptic regularity theory, we may assume that 𝑢−−→_𝑖^𝐶1𝑢, (1/((1+𝑚_𝑖𝑢_𝑖)(1+𝑘𝑣_𝑖)))⇀^𝐿2ℎ and 𝑎_𝑖→𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))]. By taking the limit in (4.12), we know that 𝑢 satisfies the following equation weakly: Δ𝑢+𝑎−𝑏𝜃_𝑐ℎ𝑢=0,‖𝑢‖_∞=1,𝑢|_𝜕Ω=0. (4.15) By Harnack's inequality, we have 𝑢>0 in Ω. Since 𝑚_𝑖‖𝑢_𝑖‖_∞→∞ and 𝑢_𝑖→𝑢, then 1/(1+𝑚_𝑖𝑢_𝑖)(1+𝑘𝑣_𝑖)=1/(1+𝑚_𝑖‖𝑢_𝑖‖_∞𝑢_𝑖)(1+𝑘𝑣_𝑖)→0 in any compact subset of Ω. Hence ℎ≡0 and 𝑎=𝜆₁, which contradicts the assumption that 𝑎≥𝜆₁+𝜖. Therefore 𝑚_𝑖‖𝑢_𝑖‖_∞ is uniformly bounded.
Let 𝑤_𝑖=𝑚_𝑖𝑢_𝑖. Then 𝑤_𝑖 satisfies Δ𝑤_𝑖+𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑤_𝑖1+𝑘𝑣_𝑖𝑤_𝑖=0,𝑤_𝑖𝜕Ω=0. (4.16) Since ‖𝑤_𝑖‖_∞=𝑚_𝑖‖𝑢_𝑖‖_∞ is bounded, due to standard elliptic regularity theory and Sobolev embedding theorems, we may assume that 𝑤−−→_𝑖^𝐶1𝑤. By letting 𝑖→∞ in (4.16), we know that 𝑤 is a nonnegative solution of (4.12). There are two possibilities as follows.(i)𝑎=𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). For this case, 𝑚_𝑖𝑢_𝑖=𝑤_𝑖→𝑤≡0. Since any positive solution of (4.12) with 𝑎=𝑎_𝑖 approaches zero when 𝑎_𝑖→𝑎, 𝑚_𝑖𝑢_𝑖 is certainly close to positive solutions of (4.12) with 𝑎=𝑎_𝑖.(ii)𝜆₁+𝜖≤𝑎<𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). In this case, we will prove that 𝑤 is a positive solution of (4.12). If not, by Harnack's inequality, we have 𝑤≡0. Let 𝑤_𝑖=𝑤_𝑖/‖𝑤_𝑖‖_∞. Then we have Δ𝑤_𝑖+𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑤_𝑖1+𝑘𝑣_𝑖𝑤_𝑖=0,𝑤_𝑖𝜕Ω=0. (4.17)Hence we may assume that 𝑤−−→_𝑖^𝐶1𝑤. By taking the limit in (4.17), we find Δ𝑤+𝑎−𝑏𝜃_𝑐1+𝑘𝜃_𝑐𝑤=0,𝑤|_𝜕Ω=0. (4.18) Since 𝑎<𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), we must have 𝑤≡0, which contradicts ‖𝑤‖_∞=lim_𝑖→∞‖𝑤_𝑖‖_∞=1. The proof is complete.

The proof of (ii) in Theorem 4.4 is more difficult. To this end, we need several lemmas. First we show that there is no positive solution of (1.2) with small 𝑢 component if 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) and 𝑚 is large.

Lemma 4.8. There exists a large 𝑀 such that if 𝑚≥𝑀, then for all 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), any positive solution (𝑢,𝑣) of (1.2) satisfies 𝑢≥𝜃_𝜆, where 𝜆=(𝜆₁+𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)))/2.

Proof. Suppose that our conclusion is not true, then there exist 𝑚_𝑖→∞,𝑎_𝑖≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), and a positive solution sequence {(𝑢_𝑖,𝑣_𝑖)} of (1.2) with (𝑎,𝑚)=(𝑎_𝑖,𝑚_𝑖) such that 𝑢_𝑖≥𝜃_𝜆 does not hold.
First, let 𝑎_𝑖→𝑎∈(𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)),∞]. Since 𝜃_𝑐≤𝑣_𝑖≤𝜃_{(𝑐+𝑑/𝑚𝑖)}, for large 𝑖, we obtain −Δ𝑢_𝑖≥𝑎+𝜆₁𝑏𝜃_𝑐/1+𝑘𝜃_𝑐2−𝑢_𝑖−𝑏𝜃_(𝑐+𝛿)1+𝑘𝜃_(𝑐+𝛿)𝑢_𝑖, (4.19) where 𝛿>0 is small such that 𝜆₁(𝑏𝜃_(𝑐+𝛿)/(1+𝑘𝜃_(𝑐+𝛿)))<(𝑎+𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)))/2, which is possible when 𝑎>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). Due to the super- and subsolution method, we obtain 𝑢_𝑖≥𝑤, where 𝑤 is a unique positive solution of −Δ𝑤=𝑎+𝜆₁𝑏𝜃_𝑐/1+𝑘𝜃_𝑐2−𝑏𝜃_(𝑐+𝛿)1+𝑘𝜃_(𝑐+𝛿)−𝑤𝑤,𝑤|_𝜕Ω=0. (4.20) Hence for large 𝑖, we get −Δ𝑢_𝑖≥𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑢_𝑖−𝑏𝑚_𝑖sup_Ω𝜃_{(𝑐+𝑑/𝑚𝑖)}𝑤𝑢_𝑖≥𝜆−𝑢_𝑖𝑢_𝑖. (4.21) Applying the super- and subsolution method again, we obtain 𝑢_𝑖≥𝜃_𝜆, which contradicts the assumption.
Secondly, we consider the case that 𝑎_𝑖→𝑎=𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). Thanks to standard elliptic regularity theory, we may assume that 𝑢_𝑖→𝑢,𝑣_𝑖→𝑣 in 𝐶¹ and 1/(1+𝑚_𝑖𝑢_𝑖)(1+𝑘𝑣_𝑖) weakly converge to ℎ in 𝐿² with 0≤ℎ≤1. Hence 𝑢 satisfies the following equation weakly: −Δ𝑢=𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑏𝑣ℎ−𝑢𝑢,𝑢|_𝜕Ω=0. (4.22) If 𝑢≥0,≢0, by Harnack's inequality, we have 𝑢>0 in Ω. Thus ℎ≡0 and 𝑢−−→_𝑖^𝐶1𝜃_{𝜆1(𝑏𝜃𝑐/(1+𝑘𝜃𝑐))}. Since 𝜃_{𝜆1(𝑏𝜃𝑐/(1+𝑘𝜃𝑐))}>𝜃_𝜆, we have 𝑢_𝑖≥𝜃_𝜆 for large 𝑖. This contradicts our assumption at the beginning of the proof. If 𝑢≡0, by letting 𝑢_𝑖=𝑢_𝑖/||𝑢_𝑖||_∞, then we know that 𝑢_𝑖 satisfies Δ𝑢_𝑖+𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖𝑢_𝑖=0,𝑢_𝑖𝜕Ω=0. (4.23) Thanks to standard elliptic regularity theory, we may assume that 𝑢−−→_𝑖^𝐶1𝑢 with ‖𝑢‖_∞=1 and 𝑢>0 in Ω. Since 𝑢−−→_𝑖^𝐶1𝑢≡0, we have 𝑣−−→_𝑖^𝐶1𝜃_𝑐. By taking the weak limit in (4.23), we have Δ𝑢+𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑏𝜃_𝑐ℎ𝑢=0,𝑢|_𝜕Ω=0. (4.24) Let Φ>0 be the positive solution to −ΔΦ+𝑏𝜃_𝑐1+𝑘𝜃_𝑐Φ=𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐Φ,Φ|_𝜕Ω=0,‖Φ‖_∞=1. (4.25) Multiplying (4.24) by Φ and integrating, we obtain 𝑏_Ω𝑢Φ𝜃_𝑐11+𝑘𝜃_𝑐−ℎ=0. (4.26) Since 0≤ℎ≤1/(1+𝑘𝜃_𝑐), we must have ℎ≡1/(1+𝑘𝜃_𝑐) and 𝑢≡Φ. Investigate the following equation for 𝑢_𝑖: Δ𝑢_𝑖+𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖𝑢_𝑖=0,𝑢_𝑖𝜕Ω=0. (4.27) Multiplying (4.27) by Φ and using (4.25), we obtain 𝑎_𝑖−𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐Ω𝑢_𝑖Φ=_Ω𝑢²_𝑖Φ+_Ω𝑏𝑢_𝑖𝑣_𝑖Φ1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖−_Ω𝑏𝜃_𝑐𝑢_𝑖Φ1+𝑘𝜃_𝑐. (4.28) After some rearrangements, we obtain 𝑎_𝑖−𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐Ω𝑢_𝑖Φ=_Ω𝑢²_𝑖Φ+𝑏_Ω𝑢_𝑖𝑣_𝑖−𝜃_𝑐Φ1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖1+𝑘𝜃_𝑐−𝑏𝑚_𝑖Ω𝜃_𝑐𝑢²_𝑖Φ1+𝑚_𝑖𝑢_𝑖1+𝑘𝜃_𝑐. (4.29) Set 𝑤_𝑖=(𝑣_𝑖−𝜃_𝑐)/||𝑢_𝑖||_∞. By the definition of 𝑣_𝑖 and 𝜃_𝑐, we have −Δ𝑤_𝑖+−𝑐+2𝜃_𝑐𝑤_𝑖=𝜃_𝑐−𝑣_𝑖𝑤_𝑖+𝑑𝑣_𝑖𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖. (4.30) Multiplying (4.30) by 𝑤_𝑖 and integrating, we obtain 𝜆₁−𝑐+2𝜃_𝑐Ω𝑤²_𝑖≤_Ω∇𝑤_𝑖²+−𝑐+2𝜃_[𝑐]𝑤²_𝑖≤𝑣_𝑖−𝜃_𝑐∞Ω𝑤²_𝑖+𝑑𝑢_𝑖∞𝑣_𝑖∞Ω𝑤_𝑖. (4.31) Since ||𝑣_𝑖−𝜃_𝑐||_∞→0, ‖𝑢_𝑖‖_∞ and ‖𝑣_𝑖‖_∞ are bounded, we know that ‖𝑤_𝑖‖₂ is bounded by Hölder inequality. Therefore from 𝐿^𝑝 estimate and Sobolev embedding theorem, it is easy to see that ||𝑤_𝑖||_∞ is bounded. Dividing (4.29) by ‖𝑢_𝑖‖²_∞, we show that 𝑎_𝑖−𝜆₁𝑏𝜃_𝑐/1+𝑘𝜃_𝑐𝑢_𝑖∞Ω𝑢_𝑖Φ=_Ω𝑢²_𝑖Φ+𝑏_Ω𝑢_𝑖𝑤_𝑖Φ1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖1+𝑘𝜃_𝑐−𝑏𝑚_𝑖Ω𝜃_𝑐𝑢²_𝑖Φ1+𝑚_𝑖𝑢_𝑖1+𝑘𝜃_𝑐. (4.32) Since 𝑢−−→_𝑖^𝐶1Φ>0,1/(1+𝑚_𝑖𝑢_𝑖)(1+𝑘𝑣_𝑖)⇀1/(1+𝑘𝜃_𝑐) weakly in 𝐿² and 𝑤_𝑖 are uniformly bounded, we obtain 𝑎_𝑖−𝜆₁𝑏𝜃_𝑐/1+𝑘𝜃_𝑐𝑢_𝑖∞→−∞ (4.33) as 𝑖→∞. Hence 𝑎_𝑖<𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) if 𝑖 is large enough, which contradicts our assumption that 𝑎_𝑖≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)) for all 𝑖. Thus, the proof is complete.

By Lemma 4.8 and a simple variant of the proof of Lemma 4.6, we immediately get the following result.

Lemma 4.9. For any given 𝐴>𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), there exists that 𝑀=𝑀(𝐴)>0 is large such that if 𝑎∈[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)),𝐴] and 𝑚≥𝑀; then any positive solution of (1.2) is nondegenerate and linearly stable.

Next we investigate the case that 𝑎 is large.

Lemma 4.10. For any 𝜖>0, there exists that 𝐴=𝐴(𝜖)>0 is large such that if 𝑚≥𝜖,𝑐≤1/𝜖 and 𝑎>𝐴, then any positive solution of (1.2) is nondegenerate and linearly stable.

Proof. Suppose the conclusion is not true. Then there exist some 𝜖₀>0,𝑚_𝑖≥𝜖₀,𝑐_𝑖≤1/𝜖₀,𝑎_𝑖→∞,Re𝜂_𝑖≤0, and (𝜙_𝑖,𝜑_𝑖)≢(0,0) with ‖𝜙_𝑖‖²₂+‖𝜑_𝑖‖²₂=1 such that Δ𝜙_𝑖+𝑎_𝑖−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖𝜙_𝑖−𝑏𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²𝜑_𝑖+𝜂_𝑖𝜙_𝑖=0,Δ𝜑_𝑖+𝑐_𝑖−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²𝜑_𝑖+𝑑𝑣_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖𝜙_𝑖+𝜂_𝑖𝜑_𝑖=0, (4.34)(4.35) where (𝑢_𝑖,𝑣_𝑖) is a positive solution of (1.2) with (𝑎,𝑐,𝑚)=(𝑎_𝑖,𝑐_𝑖,𝑚_𝑖). Since 𝑐_𝑖>𝜆₁−𝑑/𝑚_𝑖, we may assume that 𝑐_𝑖→𝑐∈[𝜆₁−𝑑/𝜖₀,(1/𝜖₀)],𝑚_𝑖→𝑚∈[𝜖₀,∞]. Let 𝛿=𝑏(1+𝑑)/(𝜖₀+𝑘(1+𝑑)). From 𝑣_𝑖≤𝜃_{(𝑐𝑖+𝑑/𝑚𝑖)}≤(1+𝑑)/𝜖₀, we have −Δ𝑢_𝑖≥𝑎_𝑖−𝑢_𝑖−𝑏𝑣_𝑖1+𝑘𝑣_𝑖𝑢_𝑖≥𝑎_𝑖−𝛿−𝑢_𝑖. (4.36) Hence 𝑢_𝑖≥𝜃_{(𝑎𝑖−𝛿)}. Due to Kato's inequality, we have −Δ𝜙_𝑖≤−Re𝜙_𝑖𝜙_𝑖Δ𝜙_𝑖≤𝑎_𝑖−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖𝜙_𝑖+𝑏𝑢_𝑖𝜑_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²+Re𝜂_𝑖𝜙_𝑖≤𝑎_𝑖−2𝑢_𝑖𝜙_𝑖+𝑏𝜑_𝑖𝑚_𝑖. (4.37) Multiplying (4.37) by |𝜙_𝑖| and integrating by parts, we obtain 𝜆₁−𝑎_𝑖+2𝜃_{(𝑎𝑖−𝛿)Ω}𝜙_𝑖²≤𝜆₁−𝑎_𝑖+2𝑢_𝑖Ω𝜙_𝑖²≤_Ω∇𝜙_𝑖²+−𝑎_𝑖+2𝑢_𝑖𝜙_𝑖²≤𝑏𝑚_𝑖Ω𝜙_𝑖𝜑_𝑖≤𝐶. (4.38) By the method of Lemma 2.2 in [18], there exists 𝑘₀∈(1,2^2/3) such that 𝜆₁−𝑎_𝑖+2𝜃_{(𝑎𝑖−𝛿)}=−𝑎_𝑖+𝜆₁2𝜃_{(𝑎𝑖−𝛿)}≥−𝑎_𝑖+𝑘₀𝑎_𝑖=𝑘−𝛿₀−1𝑎_𝑖−𝑘₀𝛿→+∞. (4.39) It follows from (4.38) and (4.39) that ‖𝜙_𝑖‖₂→0. Multiplying (4.34) by 𝜙_𝑖 and integrating, we have _Ω∇𝜙_𝑖²=_Ω𝑎_𝑖−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖𝜙_𝑖²−𝑏_Ω𝑢_𝑖𝜑_𝑖𝜙_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²+𝜂_𝑖Ω𝜙_𝑖². (4.40) Multiplying (4.35) by 𝜑_𝑖 and integrating, we have _Ω∇𝜑_𝑖²=_Ω𝑐_𝑖−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²𝜑_𝑖²+𝑑_Ω𝑣_𝑖𝜙_𝑖𝜑_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖+𝜂_𝑖Ω𝜑_𝑖². (4.41) Adding the previous two identities, we have 𝜂_𝑖=_Ω∇𝜙_𝑖²−_Ω𝑎_𝑖−2𝑢_𝑖−𝑏𝑣_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖𝜙_𝑖²+𝑏_Ω𝑢_𝑖𝜑_𝑖𝜙_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²+_Ω∇𝜑_𝑖²−_Ω𝑐_𝑖−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²𝜑_𝑖²−𝑑_Ω𝑣_𝑖𝜙_𝑖𝜑_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖. (4.42) It is easy to show that the imaginary part of the right-hand side of the previous identity is bounded. On the other hand, due to (4.38), (4.39) and the fact that ∫_Ω|(∇|𝜙_𝑖|)|²≤∫_Ω|∇𝜙_𝑖|², we have Re𝜂_𝑖 is bounded nextly. Hence 𝜂_𝑖 is bounded as we assume that Re𝜂_𝑖≤0. Thus we may assume that 𝜂_𝑖→𝜂 with Re𝜂≤0. Thanks to (4.35) and standard elliptic regularity theory, ‖𝜑_𝑖‖_𝑊2,2 is bounded. Therefore we may assume that 𝜑_𝑖→𝜑 in 𝐻¹₀. Since 𝑎_𝑖→∞,𝜃_{(𝑎𝑖−𝛿)}≤𝑢_𝑖≤𝜃_(𝑎𝑖) and 𝜃_(𝑎𝑖)/𝑎_𝑖→1, we have 𝑢_𝑖→∞ and 𝑣_𝑖→𝜃_{(𝑐+𝑑/𝑚)} in 𝐶¹. By letting 𝑖→∞ in (4.35), we know that 𝜑 satisfies the following equation weakly:𝑑Δ𝜑+𝑐+𝑚−2𝜃_{(𝑐+𝑑/𝑚)}𝜑+𝜂𝜑=0 (4.43) with Re𝜂≤0. The self-disjointness of the previous problem implies 𝜂∈𝑅. Since 𝜆₁(𝜃_{(𝑐+𝑑/𝑚)}−(𝑐+𝑑/𝑚))=0, we have 𝜂=𝜆₁2𝜃_{(𝑐+𝑑/𝑚)}−𝑑𝑐+𝑚≥𝜆₁𝜃_{(𝑐+𝑑/𝑚)}−𝑑𝑐+𝑚=0. (4.44) Hence 𝜂=0, which implies 𝑐+𝑑/𝑚=𝜆₁,𝜃_{(𝑐+𝑑/𝑚)}≡0 and 𝜑=𝛽Φ₁/‖Φ₁‖₂,|𝛽|=1. Using Kato's inequality again, we have −Δ𝜑_𝑖≤−Re𝜑_𝑖𝜑_𝑖Δ𝜑_𝑖≤𝑐_𝑖−2𝑣_𝑖+𝑑𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖²𝜑_𝑖+𝑑𝑣_𝑖𝜙_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖+Re𝜂_𝑖𝜑_𝑖. (4.45) It follows that 𝑣_𝑖 satisfies Δ𝑣_𝑖+𝑐_𝑖−𝑣_𝑖+𝑑𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖𝑣_𝑖=0,𝑣_𝑖|_𝜕Ω=0. (4.46) Multiplying (4.45) by 𝑣_𝑖, integrating by parts, and using (4.46), we obtain _Ω𝑣²_𝑖𝜑_𝑖≤𝑑_Ω𝑣²_𝑖𝜙_𝑖1+𝑚_𝑖𝑢_𝑖²1+𝑘𝑣_𝑖+Re𝜂_𝑖Ω𝑣_𝑖𝜑_𝑖≤𝑑_Ω𝑣²_𝑖𝜙_𝑖. (4.47) Set 𝑣_𝑖=𝑣_𝑖/||𝑣_𝑖||_∞. Then 𝑣_𝑖 satisfies Δ𝑣_𝑖+𝑐_𝑖−𝑣_𝑖+𝑑𝑢_𝑖1+𝑚_𝑖𝑢_𝑖1+𝑘𝑣_𝑖𝑣_𝑖=0. (4.48) Thanks to standard elliptic regularity theory, we may assume that 𝑣−−→_𝑖^𝐶1𝑣. Then 𝑣 satisfies Δ𝑑𝑣+𝑐+𝑚−𝜃_{(𝑐+𝑑/𝑚)}𝑣=0. (4.49) Since 𝑐+𝑑/𝑚=𝜆₁, we have 𝑣≡Φ₁. Dividing both sides of (4.47) by ||𝑣_𝑖||²_∞, we know that _Ω𝑣_𝑖𝑣_𝑖∞²𝜑_𝑖≤𝑑_Ω𝑣_𝑖𝑣_𝑖∞²𝜙_𝑖. (4.50) Since ‖𝜙_𝑖‖₂→0, the right-hand side of (4.50) converges to 0 by Hölder inequality. However, the previous discussion implies that ∫_Ω(𝑣_𝑖/||𝑣_𝑖||_∞)²|𝜑_𝑖|→∫_Ω(Φ³₁/||Φ₁||₂)(𝑖→∞). The contradiction completes the proof.

Proof of (ii) of Theorem 4.4. It suffices to prove the uniqueness. Investigate the following system with 𝑡∈[0,1]: Δ𝑢+𝑢𝑎−𝑢−𝑡𝑏𝑣(1+𝑚𝑢)(1+𝑘𝑣)=0,𝑥∈Ω,Δ𝑣+𝑣𝑐−𝑣+𝑡𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣)=0,𝑥∈Ω,𝑢=𝑣=0,𝑥∈𝜕Ω. (4.51) Set 𝑆={(𝑢,𝑣)∈𝑋∣𝜃_𝜆/2<𝑢<𝑎,𝜃_𝑐/2<𝑣<𝑐+𝑎𝑑} and 𝐵_𝑡∶𝑆→𝑊 by 𝐵_𝑡(𝑢,𝑣)=(−Δ+𝑃)⁻¹𝑢𝑎+𝑃−𝑢−𝑡𝑏𝑣()𝑣1+𝑚𝑢)(1+𝑘𝑣𝑐+𝑃−𝑣+𝑡𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣), (4.52) where 𝑃=max{𝑎𝑑,𝑏(𝑐+𝑎𝑑)}. Thanks to standard regularity results, we can prove that 𝐵_𝑡 is a completely continuous operator. Clearly, (𝑢,𝑣) is a positive solution of (4.51) if and only if it is a positive fixed point of 𝐵_𝑡 in 𝑆. For 𝑚≥𝑀 and 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), we first show that 𝐵_𝑡 has no fixed point on 𝜕𝑆.
We claim that any positive solution (𝑢,𝑣) of (4.51) satisfies 𝑢≥𝜃_𝜆, where 𝜆=(𝜆₁+𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)))/2. Suppose that this claim is not true. Then there exist 𝑚_𝑖→∞,𝑎_𝑖≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)),𝑡_𝑖∈[0,1], and a positive solution (𝑢_𝑖,𝑣_𝑖) of (4.51) with(𝑎,𝑚,𝑡)=(𝑎_𝑖,𝑚_𝑖,𝑡_𝑖) such that 𝑢_𝑖≥𝜃_𝜆 fails. Since the case that 𝑡_𝑖→𝑡₀≡1 is considered in Lemma 4.8, it remains to discuss the case that 𝑡_𝑖→𝑡₀∈[0,1). Since 𝑣_𝑖≤𝜃_{(𝑐+𝑑/𝑚𝑖)}, we have −Δ𝑢_𝑖≥𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑡_𝑖𝑏𝜃_{(𝑐+𝑑/𝑚𝑖)}1+𝑘𝜃_𝑐−𝑢_𝑖𝑢_𝑖≥𝜆₁𝑡𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑡₀𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑢_𝑖𝑢_𝑖, (4.53) for large 𝑖, where 𝑡∈(𝑡₀,1). Therefore 𝑢_𝑖 is a supersolution to −Δ𝜔=𝜆₁𝑡𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑡₀𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝜔𝜔,𝜔|_𝜕Ω=0. (4.54) Due to the choice of 𝑡, (4.54) has a unique positive solution 𝜔. Thus we have 𝑢_𝑖≥𝜔 for all large 𝑖. Hence,−Δ𝑢_𝑖≥𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑏𝑚_𝑖sup_Ω𝜃_{(𝑐+𝑑/𝑚𝑖)}𝜔−𝑢_𝑖𝑢_𝑖≥𝜆−𝑢_𝑖𝑢_𝑖. (4.55) By the super- and subsolution method again, we have 𝑢_𝑖≥𝜃_𝜆 for large 𝑖. This is a contradiction.
Our claim implies that 𝐵_𝑡 has no fixed point on 𝜕𝑆. Hence index_𝑊(𝐵_𝑡,𝑆)≡constant. In particular, index_𝑊𝐵₁,𝑆=index_𝑊𝐵₀,𝑆. (4.56) Since 𝐵₀ has a unique fixed point (𝜃_𝑎,𝜃_𝑐) in 𝑆 and index_𝑊(𝐵₀,(𝜃_𝑎,𝜃_𝑐))=1, we have index_𝑊(𝐵₁,𝑆)=1. From Lemmas 4.8–4.10, we see that for 𝑚≥𝑀, all fixed points of 𝐵₁ fall into 𝑆, and they are nondegenerate and linearly stable. Then by compactness, there are at most finitely many fixed points of 𝐵₁, which we denote by {(𝑢_𝑖,𝑣_𝑖)}^𝑙_𝑖=1. As shown in the proof of part (i), we have index_𝑊(𝐵₁,(𝑢_𝑖,𝑣_𝑖))=1. Using the additivity property of the index, we know that 1=index_𝑊𝐵₁,𝑆=^𝑙_𝑖=1index_𝑊𝐵₁,𝑢_𝑖,𝑣_𝑖=𝑙. (4.57) Hence for 𝑚≥𝑀 and 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)), (1.2) has a unique positive solution, and it is stable.

Our final task is to establish the exact multiplicity and stability results for large 𝑚 and 𝑎 close to 𝜆₁+𝜀 or 𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). Firstly we consider the elliptic equation (4.12), which acts as a limiting problem of (1.2) when 𝑚→∞. Applying the similar method to Lemma 2.7 in paper [18], we have the following conclusion.

Lemma 4.11. The problem (4.12) has a positive solution if and only if 𝜆₁<𝑎<𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)). Moreover, all positive solutions of (4.12) are unstable. Furthermore, there exists some 𝜖₁>0 such that if 𝑎∈(𝜆₁,𝜆₂]∪[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₁,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), then (4.12) has at most one positive solution and it is nondegenerate (if it exists).
Define 𝜖₀=min{𝜆₂,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₁}−𝜆₁, where 𝜖₁ is defined by Lemma 4.11.

Theorem 4.12. For any 𝜖∈(0,𝜖₀), one can find that 𝑀=𝑀(𝜖) is large such that if 𝑎∈(𝜆₁+𝜖,𝜆₂+𝜖₀]∪[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₁,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), and 𝑚>𝑀(𝜖), then (1.2) has exactly two positive solutions, one asymptotically stable and the other unstable.

To verify Theorem 4.12, we need some intermediate results. Theorem 4.7 has shown that (1.2) has only two types of positive solutions for when 𝑚 large and 𝑎∈[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₁,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))). In the following lemma, we will prove another result.

Lemma 4.13. There exist that 𝜖₂>0 is small and 𝑀₁>0 is large; both depend only on 𝑏,𝑐,𝑑, and 𝑘, such that if 𝑎∈[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), and 𝑚>𝑀₁, then (1.2) has exactly two positive solutions, one asymptotically stable and the other unstable.

Proof. First we prove that for large 𝑚, (1.2) has a unique asymptotically stable positive solution of type (i) in Theorem 4.7. In fact, if we choose 𝛿 small enough in Theorem 4.7, then any positive solution of (1.2) of type (i) satisfies (4.8). Hence by Lemma 4.6, they are nondegenerate and linearly stable. Now by a simple variant of the proof of part (ii) of Theorem 4.4, we find that there is only one positive solution of (1.2) satisfying type (i), and it is asymptotically stable.
Next we show that (1.2) has a unique unstable positive solution of type (ii). If we can prove this, then by Theorem 4.7, our proof of Lemma 4.13 is complete. Due to Theorem 4.7 and Lemmas 4.11, if any solution (𝑢;𝑣) of (1.1) is close to (0,𝜃_𝑐), then 𝑚𝑢 must be close to 𝑤, where 𝑤 is the unique positive solution of (4.12). Hence to prove uniqueness, it suffices to show that for 𝑎∈[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), and 𝑚>𝑀₁, there is a unique pair (𝑚𝑢,𝑣), (𝑢;𝑣) being a positive solution of (1.2), close to (𝑤,𝜃_𝑐) for certain 𝜖₂ and 𝑀₁. Set 𝑈=𝑚𝑢, 𝜌=1/𝑚, and discuss −Δ𝑈=𝑈𝑎−𝜌𝑈−𝑏𝑣(1+𝑈)(1+𝑘𝑣),𝑥∈Ω,−Δ𝑣=𝑣𝑐−𝑣+𝜌𝑑𝑈(1+𝑈)(1+𝑘𝑣),𝑥∈Ω,𝑈=𝑣=0,𝑥∈𝜕Ω. (4.58) Clearly (𝑢,𝑣) solves (1.2) if and only if (𝑚𝑢,𝑣) solves (4.58) with 𝜌=1/𝑚. Thus it suffices to prove uniqueness for (4.58). For fixed 𝜌≥0, regarding 𝑎 as a parameter, we know that (𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐)),0,𝜃_𝑐) is a simple bifurcation point of (4.58). Due to a variant of Theorem 1 in [21], there exist 𝛿₁>0 and 𝐶¹ curves Γ_𝜌=𝑎𝜌,𝑠,𝑈𝜌,𝑠,𝑣𝜌,𝑠∶0<𝑠≤𝛿₁,0≤𝜌≤𝛿₁, (4.59) such that, if 0≤𝜌≤𝛿₁, then all positive solutions of (4.58) are close to 𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐,0,𝜃_𝑐=(𝑎(0,0),𝑈(0,0),𝑣(0,0))∈Γ_𝜌. (4.60) Hence we need only to prove that these curves uniformly cover 𝑎−Range[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), for suitably chosen 𝜖₂, and for fixed 𝜖₂ and 𝜌,Γ_𝜌 cover the range only once. It is easy to obtain (see Theorem 3.9) 𝜕𝑎()𝜕𝑠0,0<0,0≤𝜌≤𝛿₁,0<𝑠≤𝛿₁. (4.61) Hence 𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑎0,𝛿₁=𝑎(0,0)−𝑎0,𝛿₁>0. (4.62) By the continuity of 𝑎(𝜌,𝑠), there exist 𝛿∈(0,𝛿₁] such that 𝜖₂=min_{0≤𝜌≤𝛿}𝜆₁𝑏𝜃_𝑐1+𝑘𝜃_𝑐−𝑎𝜌,𝛿₁>0. (4.63) Therefore, if 𝑎≥𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂, then for any 𝜌∈(0,𝛿], 𝑎(𝜌,𝛿₁)≤𝑎. This implies that for 𝜌∈(0,𝛿], Γ_𝜌 covers the 𝑎−range[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))). Moreover, since 𝜕𝑎/𝜕𝑠≠0 for 0≤𝜌,𝑠≤𝛿₁, each curve covers the range only once. By choosing 𝑀₁=1/𝛿, we get that for 𝑚>𝑀₁ and 𝑎∈([𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))), (1.2) has exactly one positive solution of type (ii) in Theorem 4.7.
It remains to show that the positive solution of (1.2) close to (0,𝜃_𝑐) is unstable. In fact, when 𝑚 is sufficiently large, applying the method of the proof in Theorem 3.9, we can show that the positive solution of (1.2) close to (0,𝜃_𝑐) is unstable. we omit the proof procedure.

Proof of Theorem 4.12. By Lemma 4.13, it suffices to establish the exact multiplicity and stability when 𝑎∈𝐼=[𝜆₁+𝜖,𝜆₁+𝜖₀]∪[𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₁,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))−𝜖₂) for any given 𝜖∈(0,𝜖₀), where 𝜖₂ is defined in Lemma 4.13
From Theorem 4.7 we know that the solutions of (1.2) for 𝑎∈[𝜆₁+𝜖,𝜆₁(𝑏𝜃_𝑐/(1+𝑘𝜃_𝑐))) and 𝑚 large are of two types, that is, types (i) or (ii). As in the proof of Lemma 4.13, we can prove that there is a unique asymptotically stable positive solution of type (i). Thus to end the proof, we need only to show that there is a unique unstable positive solution of (1.2) close to (0,𝜃_𝑐) if 𝑎∈𝐼 and 𝑚 is large. Again by Lemma 4.11, it suffices to prove that there is a unique unstable positive solution (𝑢,𝑣) of (1.2) such that (𝑚𝑢,𝑣) is close to (𝑤_𝑎,𝜃_𝑐), where 𝑤_𝑎 is the unique positive solution of (4.12) as shown in Lemma 4.11. In this connection, we investigate (4.58) with 𝑎∈𝐼 and 𝜌 small. Let 𝑎^∗∈𝐼. Since the unique solution 𝑤_𝑎∗ of (4.12) with 𝑎=𝑎^∗ is nondegenerate, then (𝑤_𝑎∗,𝜃_𝑐) is a nondegenerate solution of (4.58) with (𝑎,𝜌)=(𝑎^∗,0). Clearly, (4.58) with 𝜌>0 small is a regular perturbation of (4.58) with 𝜌=0, and the perturbation is uniform for 𝑎 in the compact set 𝐼. Thus it follows from the implicit function theorem that there exist 𝛿,𝜖>0 small such that for any 𝑎∈𝐼,0≤𝜌≤𝜖, (4.58) possesses a unique positive solution (𝑢_𝑎,𝑣_𝑎) which satisfies 𝑢_𝑎−𝑤_𝑎+𝑣_𝑎−𝜃_𝑐≤𝛿. (4.64) Set 𝑀=max{1/𝜖,𝑀(𝜖,𝛿)}, where 𝑀(𝜖,𝛿) is defined in Lemma 4.11. It is easy to see that for any 𝜖∈(0,𝜖₀), there exists 𝑀=𝑀(𝜖) such that if 𝑚≥𝑀 and 𝑎∈𝐼, then (1.2) has a unique positive solution of type (ii).
It remains to prove the instability for the unique positive solution of (1.2) of type (ii). Define 𝑇 and 𝑇₀∶𝐶₀^2,𝛼(Ω)×𝐶₀^2,𝛼(Ω)→𝐶^𝛼(Ω)×𝐶^𝛼(Ω) by 𝑇𝜉𝜂=Δ𝜉+𝑎−2𝑢−𝑏𝑣(1+𝑚𝑢)²(1+𝑘𝑣)𝜉−𝑏𝑢(1+𝑚𝑢)(1+𝑘𝑣)²𝜂Δ𝜂+𝑐−2𝑣+𝑑𝑢(1+𝑚𝑢)(1+𝑘𝑣)²𝜂+𝑑𝑣(1+𝑚𝑢)²()𝜉,𝑇1+𝑘𝑣₀𝜉𝜂=Δ𝜉+𝑎−𝑏𝜃_𝑐1+𝑤_𝑎²1+𝑘𝜃_𝑐𝜉Δ𝜂+𝑐−2𝜃_𝑐𝜂+𝑑𝜃_𝑐1+𝑤_𝑎²1+𝑘𝜃_𝑐𝜉. (4.65) It is easy to show that, as 𝑚→∞, 𝑇→𝑇₀ in the operator norm uniformly for (𝑢,𝑣) approaches (0,𝜃_𝑐) with 𝑚𝑢 close to 𝑤_𝑎 and 𝑎∈[𝜆₁+𝜖,𝜆₁+𝜖₀]. Since 0 belongs to the resolvent set of 𝑇₀ and 𝜇₀=𝜆₁−𝑎+𝑏𝜃_𝑐1+𝑤_𝑎²1+𝑘𝜃_𝑐<0 (4.66) is an eigenvalue of 𝑇₀. Due to standard perturbation theory, it is easy to get that 0 also belongs to the resolvent set of 𝑇 and that 𝑇 has an eigenvalue 𝜇 close to 𝜇₀. In particular, Re𝜇<0. This shows that for all large 𝑚, the positive solution of (1.2) close to (0,𝜃_𝑐) is nondegenerate and unstable. Thus, the proof of Theorem 4.12 is complete.

Acknowledgments

The authors would like to thank the referees for their valuable suggestions. The project supported by the Scientific Research Plan Projects of Shaanxi Education Department (no. 12JK0865) and the President Fund of Xi’an Technological University (XAGDXJJ1136).