Abstract

In this paper, we study a nonlinear system involving the fractional p-Laplacian in a unit ball and establish the radial symmetry and monotonicity of its positive solutions. By using the direct method of moving planes, we prove the following result. For , if and satisfy the following nonlinear system and are nonnegative continuous functions satisfying the following: (i) and are increasing for ; (ii) , are bounded near . Then the positive solutions must be radially symmetric and monotone decreasing about the origin.

1. Introduction

Nonlinear equations involving fractional powers of the Laplacian are currently actively studied. The fractional and nonlocal operators of elliptic type due to concrete real world have been applied in finance, thin obstacle problem, optimization, quasi-geostrophic flow, etc. In recent years, there have been tremendous interests in developing the problems related to fractional Laplacian (see in [15] and the reference therein). Fractional p-Laplacian is a generalization of the fractional Laplacian, but there are few interesting results about these nonlinear equations involving fractional p-Laplacian. In [6], the authors considered the positive solution to the following problem involving the fractional p-Laplacian:where is defined bywhere stands for the Cauchy principal value. In order to make sense the integral, we require that withThey show that if is a positive solution to (1) with , and for sufficiently small, then must be radially symmetric and monotone decreasing about some point in .

The fractional p-Laplacian is a special case of the following fully nonlinear operators :where is at least local Lipschitz continuous, , and (see [7] for the introductions of these operators, one also see [8] for systems case). In this case and .

When , the fractional p-Laplacian becomes the well-known fractional Laplacian operator . In [2], the authors considered the following system involving fractional Laplacian on the whole space:Assume that, for , are nonnegative continuous functions satisfying the following: (a) and are nondecreasing about ; (b) , are bounded near and nonincreasing with and . If and are nonnegative solutions for (5), then either and are constant or and .

Motivated by the ideas of [2, 6], our main concern in this paper is to study the following nonlinear system involving fractional p-Laplacian in a unit ball:where . We obtain the following main theorem by the direct method of moving planes.

Theorem 1. Assume that and are positive solutions of system (6); are nonnegative continuous functions satisfying the following:
(i) and are increasing about .
(ii) , are bounded near .
Then the positive solutions must be radially symmetric and monotone decreasing about the origin.

Remark 2. In [3, 9], the authors introduced the direct method of moving planes, respectively, to prove symmetry, monotonicity, and nonexistence of solutions to various differential equations on the whole space and on a half space, etc., such as ones involving fractional Laplacian in [2, 3], fractional p-Laplacian in [6], and fully nonlinear operators in [4, 8]. This direct method of moving planes has some advantages, which overcome the necessary of imposing extra assumption on the solutions when using the extension method (see also [10]) or the equivalent integral equation method (see also [11]).

The key ingredients of the direct method are maximum principle for antisymmetric functions and key boundary estimate lemma. For convenience we put both of them in the appendix.

2. Proof of Theorem 1

As usual, we choose any direction to the direction; letbe the moving planes,be the region to the left of the plane, andbe the reflection of about plane , and , .

LetdenoteThen in , we haveandwhere is valued between and ; is valued between and .

Step 1. In this step, we show that, for and sufficiently closed to , it holdsWe prove (14) by a contradiction argument, then there is at least one inequality that does not hold. Without loss of generality, we assume at some point in , then there exists some , such thatthen we will show that and derive a contradiction.

Since the fractional p-Laplacian is a special case of the fully nonlinear operators , whenwe need an analysis lemma about in [6].

Lemma 3. For , it is well-known that, by the mean value theorem, we haveThen there exists a constant , such that

Then we have

Hereand

We now estimate for to derive a contradiction when is sufficiently close to . To this end, we need to use Lemma 3. Let . Noticing that in , for some positive constant and , we havewhere is the width of the region in the -direction.

HenceTogether with (12) and , it is easy to deduce thatandand we haveFrom (24), there exists such thatSimilar to (23), we can deriveand, combining (13) and (26), we can deduce thatthenwhereWhen is sufficiently close to , becomes sufficiently small; combining assumptions (i) and (ii) in Theorem 1, we can deduce that is sufficiently small and we can let . According to (24), inequality (30) becomes

This is a contradiction with the fact that is the negative minimum of . Therefore (14) must be true for sufficiently closed to .

Step 2. Inequality (14) provides a starting point, from which we move plane toward the right as long as (14) holds to its limiting position to show that are symmetric about the limiting plane. More precisely, letWe will show thatandSuppose that , and but and , from that nonnegative functions or are positive at some point in ; combining (12), (13), and the fact that , by using the strong maximum principle of Theorem A.1, we obtain

In fact, if there exists some , such thatwe have,On the other hand,A contradiction with (38), hence (36) must be true.

By the definition of , there exists a sequence , and , such thatand, combining (23) and (24), there also exists a sequence and , such that

There is a subsequence of and that converges to some point and , respectively. And from (40), (41), and the continuity of and its derivative with respect to both and , we haveand

Then, we haveand

This will contradict with Theorem A.2. Therefore, we have Since direction can be chosen arbitrarily, we conclude that and must be radially symmetry and monotone decreasing about the origin.

This completes the proof of Theorem 1.

Appendix

Here we use the notations in the previous sections. The following theorems are the key ingredients in applying the standard moving planes method which have been explained in [6] with detailed proof.

To compare the values of and , we denote

Theorem A.1 (([6]) (maximum principle for antisymmetric functions)). Let be a bounded domain in . Assume that is lower semicontinuous on , ifThenIf at some point in , thenThere conclusions hold for unbounded region if we further assume that

Theorem A.2 (([6]) (a key boundary estimate)). Assume that , for Suppose , and , such thatLetThen

Data Availability

The data used to support the findings of this study are included within the article.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Linfen Cao participated in the method of moving plane studies in the paper; Xiaoshan Wang carried out the evaluation of inequalities and the symmetric theorem; Zhaohui Dai drafted the manuscript. All authors read and approved the final manuscript.

Acknowledgments

This work is partially supported by HASTIT (no. 15HASTIT012), NSFC (no. 11671121), and the Bureau of Foreign Experts Affairs of Henan Province.