Positive Solutions for Some Competitive Fractional Systems in Bounded Domains

Abstract

Using some potential theory tools and the Schauder fixed point theorem, we prove the existence and precise global behavior of positive continuous solutions for the competitive fractional system (−Δ_|𝐷)^𝛼/2𝑢+𝑝(𝑥)𝑢^𝜎𝑣^𝑟=0, (−Δ_|𝐷)^𝛼/2𝑣+𝑞(𝑥)𝑢^𝑠𝑣^𝛽=0 in a bounded 𝐶^1,1-domain 𝐷 in ℝ^𝑛(𝑛≥3), subject to some Dirichlet conditions, where 0<𝛼<2, 𝜎,𝛽≥1,𝑠,𝑟≥0. The potential functions 𝑝,𝑞 are nonnegative and required to satisfy some adequate hypotheses related to the Kato class of functions 𝐾_𝛼(𝐷).

1. Introduction and Statement of Main Results

Let 𝐷 be a bounded 𝐶^1,1-domain in ℝ^𝑛(𝑛≥3) and Δ_∣𝐷 be the Dirichlet Laplacian in 𝐷. The fractional power −(−Δ_∣𝐷)^𝛼/2, 0<𝛼<2, of the negative Dirichlet Laplacian is a very useful object in analysis and partial differential equations; see, for instance, [1, 2]. There is a Markov process 𝑍^𝐷_𝛼 corresponding to −(−Δ_∣𝐷)^𝛼/2 which can be obtained as follows: we first kill the Brownian motion 𝑋 at 𝜏_𝐷, the first exit time of 𝑋 from the domain 𝐷, and then we subordinate this killed Brownian motion using the 𝛼/2-stable subordinator 𝑇_𝑡 starting at zero. For more description of the process 𝑍^𝐷_𝛼 and the development of its potential theory, we refer to [3–6].

In this paper, we will exploit these potential theory tools to study the existence of positive solutions for some nonlinear systems of fractional differential equations. More precisely, we fix two positive continuous functions 𝜑 and 𝜓 on 𝜕𝐷, and we will deal with the existence of positive continuous solutions (in the sense of distributions) for the following competitive fractional system: −Δ_∣𝐷^𝛼/2𝑢+𝑝(𝑥)𝑢^𝜎𝑣^𝑟=0in𝐷,−Δ_∣𝐷^𝛼/2𝑣+𝑞(𝑥)𝑢^𝑠𝑣^𝛽=0in𝐷,lim_{𝑥→𝑧∈𝜕𝐷}𝑢(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜑(𝑧),lim_{𝑥→𝑧∈𝜕𝐷}𝑣(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜓(𝑧), (1) where 𝜎,𝛽≥1, 𝑠,𝑟≥0, 0<𝛼<2 and the nonnegative potential functions 𝑝, 𝑞 are required to satisfy some adequate hypotheses related to the Kato class of functions 𝐾_𝛼(𝐷) (see Definition 1). The function 𝑀^𝐷_𝛼1(𝑥) is defined by 𝑀^𝐷_𝛼1(𝑥)=1−𝛼/2Γ(𝛼/2)^∞₀𝑡^{−2+(𝛼/2)}1−𝑃^𝐷_𝑡1(𝑥)𝑑𝑡, (2) where (𝑃^𝐷_𝑡)_𝑡>0 is the semigroup corresponding to the killed Brownian motion upon exiting 𝐷.

We recall that in [6, Remark 3.3], the authors have proved the existence of a constant 𝐶>0 such that for each 𝑥∈𝐷, 1𝐶(𝛿(𝑥))^𝛼−2≤𝑀^𝐷_𝛼1(𝑥)≤𝐶(𝛿(𝑥))^𝛼−2, (3) where 𝛿(𝑥) denotes the Euclidian distance from 𝑥 to the boundary of 𝐷.

In the classical case (i.e., 𝛼=2), there is a large amount of literature dealing with the existence, nonexistence, and qualitative analysis of positive solutions for the problems related to (1); see for example, the papers of Cîrstea and Rădulescu [7], Ghanmi et al. [8], Ghergu and Rădulescu [9], Lair and Wood [10, 11], Mu et al. [12], and references therein. In these works, various existence results of positive bounded solutions or positive blowing up ones (called also large solutions) have been established, and a precise global behavior is given. We note also that several methods have been used to treat these systems such as sub- and supersolutions method, variational method, and topological methods. In [11], the authors studied the system (1) with 𝛼=2 in the case 𝜎=𝛽=0, 𝑠>0, 𝑟>0, and 𝑝, 𝑞 are nonnegative continuous and not necessarily radial. They showed that entire positive bounded solutions exist if 𝑝 and 𝑞 satisfy the following condition: 𝑝(𝑥)+𝑞(𝑥)≤𝐶|𝑥|^−(2+𝛿),forsomepositiveconstant𝛿and|𝑥|large. (4) These results have been extended recently by Alsaedi et al. in [13], in the case 𝛼=2, 𝜎,𝛽≥1, 𝑠>0, 𝑟>0, where the authors established the existence of a positive continuous bounded solution for (1).

In this paper, first, we aim at proving the existence and uniqueness of a positive continuous solution (in the sense of distributions) for the following scalar equation: −Δ_∣𝐷^𝛼/2𝑢+𝑝₀(𝑥)𝑢^𝛾=0in𝐷,𝑢>0in𝐷,lim_{𝑥→𝑧∈𝜕𝐷}𝑢(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜑(𝑧), (5) where 𝛾≥1 and 𝑝₀ is a nonnegative Borel measurable function in 𝐷 satisfying the following.

(𝐇₁) The function 𝑥→(𝛿(𝑥))^{(𝛼−2)(𝛾−1)}𝑝₀(𝑥)∈𝐾_𝛼(𝐷).

The class of functions 𝐾_𝛼(𝐷), is defined by means of the Green function 𝐺^𝐷_𝛼 of 𝑍^𝐷_𝛼 as follows.

Definition 1 (see [14]). A Borel measurable function 𝑞 in 𝐷 belongs to the Kato class of functions 𝐾_𝛼(𝐷) if lim_𝑟→0sup_{𝑥∈𝐷(|𝑥−𝑦|≤𝑟)∩𝐷}𝛿𝑦𝛿(𝑥)𝐺^𝐷_𝛼𝑥,𝑦𝑞𝑦𝑑𝑦=0. (6)

It has been shown in [14], that 𝑥⟶(𝛿(𝑥))^−𝜆∈𝐾_𝛼(𝐷),for𝜆<𝛼. (7) For more examples of functions belonging to 𝐾_𝛼(𝐷), we refer to [14]. Note that for the classical case (i.e., 𝛼=2), the class of functions 𝐾₂(𝐷) was introduced and studied in [15].

In order to state our existence result, we denote by 𝑀^𝐷_𝛼𝜑 (see [3]) the unique positive continuous solution of −Δ_∣𝐷^𝛼/2𝑢=0in𝐷(inthesenseofdistributions).lim_{𝑥→𝑧∈𝜕𝐷}𝑢(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜑(𝑧). (8) Using some potential theory tools and an approximating sequence, we establish the following.

Theorem 2. Under hypothesis (𝐇₁), the problem (5) has a unique positive continuous solution satisfying for each 𝑥∈𝐷, 𝑐₀𝑀^𝐷_𝛼𝜑(𝑥)≤𝑢(𝑥)≤𝑀^𝐷_𝛼𝜑(𝑥), (9) where the constant 𝑐₀∈(0,1].

Using (7), hypothesis (𝐇₁) is satisfied if 𝑝₀ verifies the following condition: there exists a constant 𝐶>0, such that for each 𝑥∈𝐷, 𝑝₀(𝑥)≤𝐶(𝛿(𝑥))^𝜏,with𝜏+(2−𝛼)𝛾−1<𝛼. (10) Next, we exploit the result of Theorem 2, to prove the existence of a positive continuous solution (𝑢,𝑣) to the system (1). To this end, we assume the following hypothesis.

(𝐇₂) The functions 𝑝, 𝑞 are two nonnegative Borel measurable functions such that 𝑥⟶(𝛿(𝑥))^{(𝛼−2)(𝜎+𝑟−1)}𝑝(𝑥)∈𝐾_𝛼(𝐷),𝑥⟶(𝛿(𝑥))^{(𝛼−2)(𝛽+𝑠−1)}𝑞(𝑥)∈𝐾_𝛼(𝐷). (11) Then, by using Schauder’s fixed point theorem, we prove the following.

Theorem 3. Under assumption (𝐇₂), the problem (1) has a positive continuous solution (𝑢,𝑣) satisfying for each 𝑥∈𝐷, 𝑐₁𝑀^𝐷_𝛼𝜑(𝑥)≤𝑢(𝑥)≤𝑀^𝐷_𝛼𝜑(𝑥),𝑐₂𝑀^𝐷_𝛼𝜓(𝑥)≤𝑣(𝑥)≤𝑀^𝐷_𝛼𝜓(𝑥), (12) where 𝑐₁∈(0,1] and 𝑐₂∈(0,1].

We note that contrary to the classical case 𝛼=2, in our situation, the solution blows up on the boundary of 𝐷.

The content of this paper is organized as follows. In Section 2, we collect some properties of functions belonging to the Kato class of functions 𝐾_𝛼(𝐷), which are useful to establish our results. Our main results are proved in Section 3.

As usual, let 𝐵⁺(𝐷) be the set of nonnegative Borel measurable functions in 𝐷. We denote by 𝐶₀(𝐷) the set of continuous functions in 𝐷 vanishing continuously on 𝜕𝐷. Note that 𝐶₀(𝐷) is a Banach space with respect to the uniform norm ‖𝑢‖_∞=sup_𝑥∈𝐷|𝑢(𝑥)|. When two positive functions 𝑓 and 𝑔 are defined on a set 𝑆, we write 𝑓≈𝑔 if the two-sided inequality (1/𝐶)𝑔≤𝑓≤𝐶𝑔 holds on 𝑆. We define the potential kernel 𝐺^𝐷_𝛼 of 𝑍^𝐷_𝛼 by 𝐺^𝐷_𝛼𝑓(𝑥)∶=_𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑓𝑦𝑑𝑦,for𝑓∈𝐵⁺(𝐷),𝑥∈𝐷. (13) Finally, let us recall some potential theory tools that are needed, and we refer to [14, 16, 17] for more details. For 𝑞∈𝐵⁺(𝐷), we define the kernel 𝑉_𝑞 on 𝐵⁺(𝐷) by 𝑉_𝑞𝑓(𝑥)∶=^∞₀𝐸^𝑥𝑒^{−∫𝑡0𝑞(𝑍𝐷𝛼(𝑠))𝑑𝑠}𝑓𝑍^𝐷_𝛼(𝑡)𝑑𝑡,𝑥∈𝐷, (14) with 𝑉₀∶=𝑉=𝐺^𝐷_𝛼, where 𝐸^𝑥 stands for the expectation with respect to 𝑍^𝐷_𝛼 starting from 𝑥. If 𝑞 satisfies 𝑉𝑞<∞, we have the following resolvent equation: 𝑉=𝑉_𝑞+𝑉_𝑞𝑞𝑉=𝑉_𝑞+𝑉𝑞𝑉_𝑞. (15) In particular, if 𝑢∈𝐵⁺(𝐷) is such that 𝑉(𝑞𝑢)<∞, then we have 𝐼−𝑉_𝑞𝑞.𝐼+𝑉𝑞.𝑢=𝐼+𝑉𝑞.𝐼−𝑉_𝑞𝑞.𝑢=𝑢. (16)

2. The Kato Class of Functions 𝐾_𝛼(𝐷)

Proposition 4 (see [14]). Let𝑞be a function in𝐾_𝛼(𝐷), then we have the following.(i)𝑎_𝛼(𝑞)∶=sup_{𝑥,𝑦∈𝐷}∫_𝐷(𝐺^𝐷_𝛼(𝑥,𝑧)𝐺^𝐷_𝛼(𝑧,𝑦)/𝐺^𝐷_𝛼(𝑥,𝑦))|𝑞(𝑧)|𝑑𝑧<∞. (ii)Letℎbe a positive excessive function on𝐷with respect to𝑍^𝐷_𝛼. Then, we have_𝐷𝐺^𝐷_𝛼𝑥,𝑦ℎ𝑦𝑞𝑦𝑑𝑦≤𝑎_𝛼𝑞ℎ(𝑥). (17) Furthermore, for each𝑥₀∈𝐷, we havelim_𝑟→0sup_𝑥∈𝐷1ℎ(𝑥)_{𝐵(𝑥0,𝑟)∩𝐷}𝐺^𝐷_𝛼𝑥,𝑦ℎ𝑦𝑞𝑦𝑑𝑦=0. (18)(iii)The function  𝑥→(𝛿(𝑥))^𝛼−1𝑞(𝑥)is in 𝐿¹(𝐷).

The next two lemmas will play a special role.

Lemma 5. Let 𝑞 be a nonnegative function in 𝐾_𝛼(𝐷) and ℎ be a positive finite excessive function on 𝐷 with respect to 𝑍^𝐷_𝛼. Then, for all 𝑥∈𝐷, we have exp−𝑎_𝛼𝑞ℎ(𝑥)≤ℎ(𝑥)−𝑉_𝑞𝑞ℎ(𝑥)≤ℎ(𝑥). (19)

Proof. Let ℎ be a positive finite excessive function on 𝐷 with respect to 𝑍^𝐷_𝛼. Then, by [18, Chapter II, proposition 3.11], there exists a sequence (𝑓_𝑘)_𝑘 of nonnegative measurable functions in 𝐷 such that ℎ=sup_𝑘𝑉𝑓_𝑘. Let 𝑥∈𝐷 and 𝑘∈ℕ such that 0<𝑉𝑓_𝑘<∞. Consider 𝜃(𝑡)=𝑉_𝑡𝑞𝑓_𝑘(𝑥), for 𝑡≥0. Then, by (14), the function 𝜃 is completely monotone on [0,∞), and so from the Hölder inequality and [19, Theorem 12a], the function log𝜃 is convex on [0,∞). This implies that 𝜃(0)(1)−𝜃≤𝜃exp^′(0)𝜃(0). (20) That is 𝑉𝑓_𝑘(𝑥)≤𝑉_𝑞𝑓_𝑘(𝑥)𝑉exp𝑞𝑉𝑓_𝑘(𝑥)𝑉𝑓_𝑘(𝑥). (21) Hence, it follows from (17) that 𝑉𝑓_𝑘(𝑥)≤𝑉_𝑞𝑓_𝑘(𝑥)exp𝑎_𝛼𝑞. (22) Consequently, from (15) we obtain exp−𝑎_𝛼𝑞𝑉𝑓_𝑘(𝑥)≤𝑉𝑓_𝑘(𝑥)−𝑉_𝑞𝑞𝑉𝑓_𝑘(𝑥)≤𝑉𝑓_𝑘(𝑥). (23) The result holds by letting 𝑘→∞.

Lemma 6. Let 𝑞 be a nonnegative function in 𝐾_𝛼(𝐷), then the family of functions Λ_𝑞=1𝑀^𝐷_𝛼𝜑(𝑥)_𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑𝑦𝑓𝑦𝑑𝑦,𝑓≤𝑞 (24) is uniformly bounded and equicontinuous in 𝐷. Consequently, Λ_𝑞 is relatively compact in 𝐶₀(𝐷).

Proof. Taking ℎ≡𝑀^𝐷_𝛼𝜑 in (17), we deduce that for |𝑓|≤𝑞 and 𝑥∈𝐷, we have _𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)𝑀^𝐷_𝛼𝜑𝑦𝑓𝑦≤𝑑𝑦_𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦≤𝑎_𝛼𝑞<∞. (25) So, the family Λ_𝑞 is uniformly bounded.
Next, we aim at proving that the family Λ_𝑞 is equicontinuous in 𝐷.
First, we recall the following interesting sharp estimates on 𝐺^𝐷_𝛼, which is proved in [5]: 𝐺^𝐷_𝛼𝑥,𝑦≈𝑥−𝑦^𝛼−𝑛𝛿(𝑥)𝛿𝑦min1,𝑥−𝑦². (26) Let 𝑥₀∈𝐷 and 𝜀>0. By (18), there exists 𝑟>0 such that sup_𝑧∈𝐷1𝑀^𝐷_𝛼𝜑(𝑧)_{𝐵(𝑥0,2𝑟)∩𝐷}𝐺^𝐷_𝛼𝑧,𝑦𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝜀𝑑𝑦≤2. (27) If 𝑥₀∈𝐷 and 𝑥,𝑥^′∈𝐵(𝑥₀,𝑟)∩𝐷, then for |𝑓|≤𝑞, we have _𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)𝑀^𝐷_𝛼𝜑𝑦𝑓𝑦𝑑𝑦−_𝐷𝐺^𝐷_𝛼𝑥^′,𝑦𝑀^𝐷_𝛼𝜑𝑥^′𝑀^𝐷_𝛼𝜑𝑦𝑓𝑦≤𝑑𝑦_𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)−𝐺^𝐷_𝛼𝑥^′,𝑦𝑀^𝐷_𝛼𝜑𝑥^′𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦≤2sup_{𝑧∈𝐷𝐵(𝑥0,2𝑟)∩𝐷}1𝑀^𝐷_𝛼𝜑(𝑧)𝐺^𝐷_𝛼𝑧,𝑦𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦+𝑑𝑦_{(|𝑥0−𝑦|≥2𝑟)∩𝐷}𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)−𝐺^𝐷_𝛼𝑥^′,𝑦𝑀^𝐷_𝛼𝜑𝑥^′𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦≤𝜀+_{(|𝑥0−𝑦|≥2𝑟)∩𝐷}𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)−𝐺^𝐷_𝛼𝑥^′,𝑦𝑀^𝐷_𝛼𝜑𝑥^′𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦. (28) On the other hand, for every 𝑦∈𝐵^𝑐(𝑥₀,2𝑟)∩𝐷 and 𝑥,𝑥^′∈𝐵(𝑥₀,𝑟)∩𝐷, by using (26) and the fact that 𝑀^𝐷_𝛼𝜑(𝑧)≈(𝛿(𝑧))^𝛼−2, we have 1𝑀^𝐷_𝛼𝜑(𝑥)𝐺^𝐷_𝛼−1𝑥,𝑦𝑀^𝐷_𝛼𝜑𝑥^′𝐺^𝐷_𝛼𝑥^′,𝑦𝑀^𝐷_𝛼𝜑𝑦≤𝐶𝛿𝑦^𝛼−1. (29) Now since 𝑥→(1/𝑀^𝐷_𝛼𝜑(𝑥))𝐺^𝐷_𝛼(𝑥,𝑦) is continuous outside the diagonal and 𝑞∈𝐾_𝛼(𝐷), we deduce by the dominated convergence theorem and Proposition 4 (iii) that _{(|𝑥0−𝑦|≥2𝑟)∩𝐷}𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)−𝐺^𝐷_𝛼𝑥^′,𝑦𝑀^𝐷_𝛼𝜑𝑥^′𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦⟶0as𝑥−𝑥^′⟶0. (30) If 𝑥₀∈𝜕𝐷 and 𝑥∈𝐵(𝑥₀,𝑟)∩𝐷, then we have _𝐷𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)𝑀^𝐷_𝛼𝜑𝑦𝑓𝑦≤𝜀𝑑𝑦2+_{(|𝑥0−𝑦|≥2𝑟)∩𝐷}𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦. (31) Now, since 𝐺^𝐷_𝛼(𝑥,𝑦)/𝑀^𝐷_𝛼𝜑(𝑥)→0 as |𝑥−𝑥₀|→0, for |𝑥₀−𝑦|≥2𝑟, then by the same argument as above, we get _{(|𝑥0−𝑦|≥2𝑟)∩𝐷}𝐺^𝐷_𝛼𝑥,𝑦𝑀^𝐷_𝛼𝜑(𝑥)𝑀^𝐷_𝛼𝜑𝑦𝑞𝑦𝑑𝑦⟶0as𝑥−𝑥₀⟶0. (32) Consequently, by Ascoli’s theorem, we deduce that Λ_𝑞 is relatively compact in 𝐶₀(𝐷).

3. Proofs of Theorems 2 and 3

The next Lemma will be used for uniqueness.

Lemma 7 (see [14, Proposition 6]). Let ℎ∈𝐵⁺(𝐷) and 𝜐 be a nonnegative excessive function on 𝐷 with respect to 𝑍^𝐷_𝛼. Let 𝑧 be a Borel measurable function in 𝐷 such that 𝑉(ℎ|𝑧|)<∞ and 𝜐=𝑧+𝑉(ℎ𝑧). Then, 𝑧 satisfies 0≤𝑧≤𝜐. (33)

Proof of Theorem 2. Let 𝜑 be a positive continuous function on 𝜕𝐷. We recall that on 𝐷, we have 𝑀^𝐷_𝛼𝜑(𝑥)≈𝑀^𝐷_𝛼1(𝑥)≈(𝛿(𝑥))^𝛼−2. (34) Let 𝑝₀=𝛾(𝑀^𝐷_𝛼𝜑)^𝛾−1𝑝₀ and put 𝑐₀=𝑒^−𝑎𝛼(𝑝₀₎, where 𝑎_𝛼(𝑝₀) is given by Proposition 4(i). Since by (𝐇₁), 𝑝₀∈𝐾_𝛼(𝐷), it follows from Proposition 4 that 𝑉(𝑝₀)≤𝑎_𝛼(𝑝₀)<∞. Define the nonempty closed bounded convex Λ by Λ=𝜔∈𝐵⁺(𝐷)∶𝑐₀≤𝜔≤1. (35) Let 𝑇 be the operator defined on Λ by 1𝑇𝜔∶=1−𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝑀^𝐷_𝛼𝜑+1𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝜔𝑀^𝐷_𝛼𝜑−𝑝₀𝜔𝑀^𝐷_𝛼𝜑^𝛾. (36) We claim that 𝑇 maps Λ to itself. Indeed, for each 𝜔∈Λ, we have 1𝑇𝜔≤1−𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝜔𝑀^𝐷_𝛼𝜑^𝛾≤1. (37) On the other hand, since the function 𝑝₀𝜔𝑀^𝐷_𝛼𝜑−𝑝₀(𝜔𝑀^𝐷_𝛼𝜑)^𝛾≥0, we deduce by Lemma 5 with ℎ=𝑀^𝐷_𝛼𝜑, that 𝑇𝜔≥1−(1/𝑀^𝐷_𝛼𝜑)𝑉_𝑝0(𝑝₀𝑀^𝐷_𝛼𝜑)≥𝑐₀. Hence, 𝑇Λ⊂Λ. Next, we aim at proving that 𝑇 is nondecreasing on Λ. To this end, we let 𝜔₁, 𝜔₂∈Λ such that 𝜔₁≤𝜔₂. Using the fact that the function 𝑡→𝛾𝑡−𝑡^𝛾 is nondecreasing on [0,1], we deduce that 𝑇𝜔₂−𝑇𝜔₁=1𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝜔₂𝑀^𝐷_𝛼𝜑−𝑝₀𝜔₂𝑀^𝐷_𝛼𝜑^𝛾−1𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝜔₁𝑀^𝐷_𝛼𝜑−𝑝₀𝜔₁𝑀^𝐷_𝛼𝜑^𝛾=1𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝑀^𝐷_𝛼𝜑^𝛾𝛾𝜔₂−𝜔^𝛾₂−𝛾𝜔₁−𝜔^𝛾₁≥0. (38) Next, we define the sequence (𝜔_𝑘)_𝑘≥0 by 𝜔₀1=1−𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝑀^𝐷_𝛼𝜑,𝜔_𝑘+1=𝑇𝜔_𝑘. (39) Clearly 𝜔₀∈Λ and 𝜔₁=𝑇𝜔₀≥𝜔₀. Thus, from the monotonicity of 𝑇, we deduce that 𝑐₀≤𝜔₀≤𝜔₁≤⋯≤𝜔_𝑘≤1. (40) So, the sequence (𝜔_𝑘)_𝑘≥0 converges to a measurable function 𝜔∈Λ. Therefore, by applying the monotone convergence theorem, we obtain 1𝜔=1−𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝑀^𝐷_𝛼𝜑+1𝑀^𝐷_𝛼𝜑𝑉_𝑝0𝑝₀𝜔𝑀^𝐷_𝛼𝜑−𝑝₀𝜔𝑀^𝐷_𝛼𝜑^𝛾. (41) Put 𝑢=𝜔𝑀^𝐷_𝛼𝜑. Then, we have 𝑢=𝑀^𝐷_𝛼𝜑−𝑉_𝑝0𝑝₀𝑀^𝐷_𝛼𝜑+𝑉_𝑝0𝑝₀𝑢−𝑝₀𝑢^𝛾 (42) or equivalently 𝑢−𝑉_𝑝0𝑝₀𝑢=𝑀^𝐷_𝛼𝜑−𝑉_𝑝0𝑝₀𝑀^𝐷_𝛼𝜑−𝑉_𝑝0𝑝₀𝑢^𝛾. (43) Observe that by Proposition 4(ii), we have 𝑉(𝑝₀𝑢)<∞. So, applying the operator (𝐼+𝑉(𝑝₀.)) on both sides of (43), we deduce by using (15) and (16) that 𝑢=𝑀^𝐷_𝛼𝜑−𝑉𝑝₀𝑢^𝛾. (44) Now, using (𝐇₁) and similar argument as in the proof of Lemma 6, we prove that 𝑥→(1/𝑀^𝐷_𝛼𝜑)𝑉(𝑝₀𝑢^𝛾)∈𝐶₀(𝐷). So, 𝑢 is a continuous function in 𝐷, and 𝑢 is a solution of (5). It remains to prove the uniqueness of such a solution. Let 𝑢 be a continuous solution of (5). Since the function 𝑥→𝑢(𝑥)/𝑀^𝐷_𝛼1(𝑥) is continuous and positive in 𝐷 such that lim_{𝑥→𝑧∈𝜕𝐷}(𝑢(𝑥)/𝑀^𝐷_𝛼1(𝑥))=𝜑(𝑧), it follows that 𝑢(𝑥)≈𝑀^𝐷_𝛼1(𝑥)≈𝑀^𝐷_𝛼𝜑(𝑥). Then, by using this fact and Lemma 6, we have −Δ_∣𝐷^𝛼/2𝑢+𝑉𝑝₀𝑢^𝛾=0in𝐷,lim_{𝑥→𝑧∈𝜕𝐷}𝑢+𝑉𝑝₀𝑢^𝛾(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜑(𝑧)in𝐷. (45) So, from the uniqueness of the problem (8) (see [3]), we deduce that 𝑢+𝑉𝑝₀𝑢^𝛾=𝑀^𝐷_𝛼𝜑in𝐷. (46) It follows that if 𝑢 and 𝑣 are two continuous solution of (5), then 𝑧=𝑣−𝑢 satisfies 𝑧+𝑉𝑝₀ℎ𝑧=0in𝐷, (47) where ℎ is the nonnegative measurable function defined in 𝐷 by ℎ(𝑥)=𝑣^𝛾−𝑢^𝛾(𝑥)(𝑥),(𝑥)(𝑥).𝑣−𝑢,if𝑢≠𝑣0,if𝑢=𝑣 (48) Since 𝑉(𝑝₀ℎ|𝑧|)<∞, we deduce by Lemma 7 that 𝑧=0, and so 𝑢=𝑣.

Proof of Theorem 3. Let 𝑝=𝜎(𝑀^𝐷_𝛼𝜑)^𝜎−1(𝑀^𝐷_𝛼𝜓)^𝑟𝑝 and 𝑞=𝛽(𝑀^𝐷_𝛼𝜓)^𝛽−1(𝑀^𝐷_𝛼𝜑)^𝑠𝑞.
Put 𝑐₁=𝑒^−𝑎𝛼(𝑝⁾, 𝑐₂=𝑒^−𝑎𝛼(𝑞₎. Note that from (𝐇₂) and Proposition 4, we have 𝑎_𝛼(𝑝)<∞ and 𝑎_𝛼(𝑞)<∞. Consider the nonempty closed convex set Γ defined by Γ=𝑦,𝑧∈𝐶𝐷×𝐶𝐷∶𝑐₁≤𝑦≤1,𝑐₂≤𝑧≤1. (49) Let 𝑇 be the operator defined on Γ by 𝑇(𝑦,𝑧)∶=(𝜔,𝜃), such that (𝑢=𝜔𝑀^𝐷_𝛼𝜑,𝑣=𝜃𝑀^𝐷_𝛼𝜓) is the unique positive continuous solution of the following problem: −Δ_∣𝐷^𝛼/2𝑢+𝑀^𝐷_𝛼𝜓^𝑟𝑧^𝑟𝑝(𝑥)𝑢^𝜎=0in𝐷,−Δ_∣𝐷^𝛼/2𝑣+𝑀^𝐷_𝛼𝜑^𝑠𝑦^𝑠𝑞(𝑥)𝑣^𝛽=0in𝐷,lim_{𝑥→𝑧∈𝜕𝐷}𝑢(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜑(𝑧),lim_{𝑥→𝑧∈𝜕𝐷}𝑣(𝑥)𝑀^𝐷_𝛼1(𝑥)=𝜓(𝑧). (50) According to Theorem 2, we have 1𝜔=1−𝑀^𝐷_𝛼𝜑𝑉𝑧^𝑟𝜔^𝜎𝑀^𝐷_𝛼𝜓^𝑟𝑀^𝐷_𝛼𝜑^𝜎𝑝1𝜃=1−𝑀^𝐷_𝛼𝜓𝑉𝑦^𝑠𝜔^𝛽𝑀^𝐷_𝛼𝜑^𝑠𝑀^𝐷_𝛼𝜓^𝛽𝑞. (51) Moreover, we have 𝑐₁≤𝜔≤1 and 𝑐₂≤𝜃≤1 and by Lemma 6, 𝑇(Γ) is equicontinuous on 𝐷. Since 𝑇(Γ) is also bounded, then we deduce that 𝑇(Γ) is relatively compact in 𝐶(𝐷)×𝐶(𝐷). This implies in particular that 𝑇(Γ)⊂Γ.
Next, we shall prove the continuity of the operator 𝑇 in Γ in the supremum norm. Let (𝑦_𝑘,𝑧_𝑘)_𝑘 be a sequence in Γ which converges uniformly to a function (𝑦,𝑧) in Γ. Put (𝜔_𝑘,𝜃_𝑘)=𝑇(𝑦_𝑘,𝑧_𝑘) and (𝜔,𝜃)=𝑇(𝑦,𝑧). Then, we have 𝜔_𝑘=1−𝜔𝑀^𝐷_𝛼𝜑𝑉𝑧^𝑟𝜔^𝜎𝑀^𝐷_𝛼𝜓^𝑟𝑀^𝐷_𝛼𝜑^𝜎𝑝−1𝑀^𝐷_𝛼𝜑𝑉𝑧^𝑟_𝑘𝜔^𝜎_𝑘𝑀^𝐷_𝛼𝜓^𝑟𝑀^𝐷_𝛼𝜑^𝜎𝑝≤1𝜎𝑀^𝐷_𝛼𝜑𝑉𝑧^𝑟𝜔^𝜎−𝑧^𝑟_𝑘𝜔^𝜎_𝑘𝑀^𝐷_𝛼𝜑𝑝. (52) Using the fact that |𝑧^𝑟𝜔^𝜎−𝑧^𝑟_𝑘𝜔^𝜎_𝑘|≤2 and that 𝑝∈𝐾_𝛼(𝐷), we deduce by Proposition 4 and the dominated convergence theorem, that 𝜔_𝑘→𝜔 as 𝑘→∞. Similarly, we prove that 𝜃_𝑘→𝜃 as 𝑘→∞. So, 𝑇(𝑦_𝑘,𝑧_𝑘)→𝑇(𝑦,𝑧) as 𝑘→∞. Since 𝑇(Γ) is relatively compact in 𝐶(𝐷)×𝐶(𝐷), we deduce that 𝑇𝑦_𝑘,𝑧_𝑘−𝑇𝑦,𝑧_∞⟶0as𝑘⟶∞. (53) From the Schauder fixed point theorem there exists (𝑦,𝑧)∈Γ such that 𝑇(𝑦,𝑧)=(𝑦,𝑧) or equivalently 𝑢=𝑀^𝐷_𝛼𝜑−𝑉𝑝𝑢^𝜎𝑣^𝑟,𝑣=𝑀^𝐷_𝛼𝜓−𝑉𝑞𝑢^𝑠𝑣^𝛽, (54) where (𝑢,𝑣)=(𝑦𝑀^𝐷_𝛼𝜑,𝑧𝑀^𝐷_𝛼𝜓). The pair (𝑢,𝑣) is a required solution of (1) in the sense of distributions. This completes the proof.

Acknowledgment

The research of Imed Bachar is supported by NPST Program of King Saud University; Project no. 11-MAT1716-02.