#### Abstract

Nonexistence and radial symmetry of positive solutions for a class of semilinear elliptic systems are considered via the method of moving spheres.

#### 1. Introduction

In this paper we consider the more general semilinear elliptic system

where and () are nonnegative constants. The question is to determine for which values of the exponents and the only nonnegative solution of (1.1) is . The solution here is taken in the classical sense, that is, . In the case of the Emden-Fowler equation

When , it has been proved in  that the only solution of (1.2) is . In dimension , a similar conclusion holds for . It is also well known that in the critical case, , problem (1.2) has a two-parameter family of solutions given by

where with and . If , and , using Pokhozhaev's second identity, Chen and Lu ([2, Theorem ]) have proved that problem (1.1) has no positive radial solutions with . Suppose that and satisfy and other related conditions, using the method of integral relations, Mitidieri ([3, Theorem ]) has proved that problem (1.1) has no positive solutions of with . In present paper, we study problem (1.1) by virtue of the method of moving spheres and obtain the following theorems of nonexistence and radial symmetry of positive solutions.

Theorem 1.1. Suppose that , but and are not equal to zero at the same time. Moreover, with , but and are not both equal to , then Problem (1.1) has no positive solution of .

Theorem 1.2. Suppose that , and , then the positive solution of (1.1) is of the form (1.3), that is, for some , where and satisfy the following equalities:

Remark 1.3. Obviously Theorem 1.1 contains new region of and which can not be covered by [2, Theorem ] and [3, Theorem ]. Moreover, Theorem 1.2 gives the exact forms of positive solutions of .

There are some related works about problem (1.1). For and , Figueiredo and Felmer (see ) proved Theorem 1.1 using the moving plane method and a special form of the maximum principle for elliptic systems. Busca and Manásevich obtained a new result (see [5, Theorem ]) using the same method as in . It allows and to reach regions where one of the two exponents is supercritical. In , Zhang et al. first introduced the Kelvin transforms and gave a different proof of Theorem 1.1 in  using the method of moving spheres. This approach was suggested in , while Li and Zhang who had made significant simplifications prove some Liouville theorems for a single equation in . In this paper, we consider the general case of nonlinearities and do not need the maximum principle for elliptic systems. Moreover, the exact form of positive solution is proved in Theorem 1.2. If we can find a proper transforms instead of the Kelvin transforms, we suspect that [5, Theorem ] can also be proved via the method of moving spheres. We leave this to the interested readers.

Let us emphasize that considerable attention has been drawn to Liouville-type results and existence of positive solutions for general nonlinear elliptic equations and systems, and that numerous related works are devoted to some of its variants, such as more general quasilinear operators and domains. We refer the interested reader to , and some of the references therein. We refer the interested reader to [16, 17].

#### 2. Preliminaries and Moving Spheres

To prove Theorems 1.1 and 1.2, we will use the method of moving spheres. We first prove a number of lemmas as follows. For and , let us introduce the Kelvin transforms

which are defined for . For any , one verifies that and satisfy the system

Our first lemma says that the method of moving spheres can get started.

Lemma 2.1. For every , there exists such that and , for all and .

Proof. Without loss of generality we may take . We use and to denote and , respectively. Clearly, there exists such that Consequently, By the superharmonicity of and the maximum principle (see [4, Corollary ]), Let Then for every , and , we have It follows from (2.4), (2.5), and (2.7) that for every , Similarly, there exists such that for every , we obtain We can choose .

Set, for ,

By Lemma 2.1, and are well defined and for . Let , then we have the following

Lemma 2.2. If for some , then and on .

Lemma 2.3. If for some , then for all .

Proof of Lemma 2.2. Without loss of generality, we assume that and take and let and , and . We wish to show and in . Clearly, it suffices to show We first prove . We know from the definition of that In view of (1.1), a simple calculation yields Therefore, If on , we stop. Otherwise, by the Hopf lemma and the compactness of , we have By the continuity of , there exists such that Consequently, since on , we have
Set . It follows from the superharmonicity of that
Therefore,
By the uniform continuity of on , there exists such that for all ,
It follows from (2.19) and the above inequality that Estimates (2.17) and (2.21) violate the definition of .
From and (2.14), we easily know that in . Lemma 2.2 is proved.

Proof of Lemma 2.3. Since , we have It follows that On the other hand, if for some , then, by Lemma 2.2, which is a contradiction. Similarly, we also obtain a contradiction for .

#### 3. Proofs of Theorems 1.1 and 1.2

In this section we first present two calculus lemmas taken from  (see also ).

Lemma 3.1 (See [8, Lemma ]). Let . Suppose that for every , there exists such that Then for some

Lemma 3.2 (See [8, Lemma ]). Let . Assume that Then

Proof of Theorem 1.1. We first claim that for all . We prove it by contradiction argument. If for some , then by Lemma 2.2, and on But looking at equations in system (2.2) we realize that this is impossible. Therefore, This, by Lemma 3.2, implies that . From system (1.1) we know that it is also impossible.

Proof of Theorem 1.2. We first claim that for all . We prove it by contradiction argument. If for some , then by Lemma 2.3, for all , that is, This, by Lemma 3.2, implies that , a contradiction to (1.1). Therefore, it follows from Lemma 2.2 that for every , there exists such that and . Then by Lemma 3.1, for some and some , Theorem 1.2 follows from the above and the fact that is a solution of (1.1).

#### Acknowledgment

This paper is supported by Youth Foundation of NSFC (no. 10701061).