1. Introduction and the Main Results

In this paper, we consider the existence of homoclinic orbits for the following Hamiltonian system: where , is a symmetric matrix-valued function, and is superquadratic both around 0 and at infinity in .

A solution of (1.1) is called to be homoclinic to 0 if and as .

In recent years, the existence and multiplicity of homoclinic orbits for Hamiltonian systems have been investigated in many papers via variational methods. See, for example, [1–7] for the second-order systems and [8–12] for the first-order systems. We note that in most of the papers on the first order system (1.1) it was assumed that

Thus, if we let denote the spectrum of , means that is independent of and there is such that Consequently, the operator is a homeomorphism for all . This is important for the variational arguments. Later in [13], Ding considered the case that depends periodically on . He made assumptions on such that 0 lies in a gap of . If additionally is periodic in and satisfies some superquadratic or asymptotically quadratic conditions in at infinity, then infinitely many homoclinic orbits were obtained.

If , then the problem is quite different in nature since the operator cannot lead the behavior at of the equation. Ding and Willem considered this case in [14]. They assumed that

Under , 0 may belong to continuous spectrum of . The authors managed to construct an appropriate Banach space, on which some embedding results necessary for variational arguments were obtained. Using a generalized linking theorem developed by Kryszewski and Szulkin in [15], they got one homoclinic orbit of (1.1). Later, Ding and Girardi obtained infinitely many homoclinic orbits in [16] under the conditions of [14] with an additional evenness assumption on . Note that in both papers satisfies a condition of the type of Ambrosetti-Rabinowitz (see [17]), that is,

The (A-R) condition is essential to prove the Palais-Smale condition since the variational functional is strongly indefinite and . The argument of Palais-Smale condition is rather technical and not standard without the (A-R) condition. In this paper, we consider the existence of solutions of (1.1) under without the (A-R) condition on .

We observed that just recently some abstract linking theorems were developed by Bartsch and Ding in [18]. These theorems are impactful to study the existence and multiplicity of solutions for the strongly indefinite problem. Many new results have been obtained by these theorems based on the use of sequence. See [19–21] for applications of these ideas. Note that in [19–21] 0 either is not a spectral point or is at most an isolated eigenvalue of finite multiplicity. Thus condition was checked by virtue of some very technical analysis. However, if , then we can find a sequence with and . Thus the operator cannot lead the behavior at 0 of the equation. Consequently, besides condition, it seems also hard to check the following condition necessary for the linking theorems in [19–21]:

Our work benefits from [14] and some weak linking theorem recently developed by Schechter and Zou in [22]. This theorem permits us first to study a sequence of approximating problems for (the initial problem corresponds to ) for which a bounded Palais-Smale sequence of is given for almost each . Then by monotonicity, we find a sequence of and such that , and . Since the sequence consists of critical points of , then its boundedness can be checked. Consequently one solution of (1.1) is obtained. The idea of first studying approximating problems for which the existence of a bounded Palais-Smale sequence is given freely and then proving that the sequence of approximated critical points is bounded was originally introduced in [23]. See also [24].

We make the following assumptions.

Theorem 1.1. Let , be satisfied, then (1.1) has at least one homoclinic orbit.

Remark 1.2. We can easily check that the (A-R) condition implies and . But the converse proposition is not true. See the following example: where , (see [25] or [26] for details).

If , , with satisfying

Theorem 1.3. Let and be satisfied, then (1.1) has at least one homoclinic orbit.

This paper is organized as follows. In Section 2 we will construct some appropriate variational space and obtain some embedding results necessary for our variational arguments. In Section 3 we will recall a weak linking theorem, by which we will give the proof of Theorems 1.1 and 1.3 in Section 4.

2. Some Embedding Results

In what follows, by we denote the usual -norm and by the usual -inner product. A standard Floquet reduction argument shows that (see [14]).

Let be the spectral family of . possesses the polar decomposition with . By , is at most a continuous spectrum of . has an orthogonal decomposition

where .

Let denote the domain of and let be the space of the completion of under the norm

becomes a Hilbert space under the inner product

possesses an orthogonal decomposition

where

Under , it is easy to check

Therefore, can be embedded continuously into for any and compactly into for any .

For any , set and Then on , we also have and the same embedding conclusion as that of .

Let where stands for the closure of in .

For , let be the completion of under the norm and let denote the completion of with respect to the norm . Then is a closed subspace of and possesses the following decomposition:

Moreover, is orthogonal to with respect to .

Let be the completion of under the norm . The following result holds true.

Lemma 2.1 (see [14]). Under , has the direct sum decomposition and is embedded continuously in for any and compactly in   for any .

3. A Weak Linking Theorem

In this section we state some weak linking theorem due to [22] which was first built in a Hilbert space. This theorem is still true in a reflexive Banach space (cf. Willem and Zou [25]).

Let be a reflexive Banach space with norm and possess a direct sum decomposition , where is a closed and separable subspace. Since is separable, we can define a new norm satisfying , such that the topology induced by this norm is equivalent to the weak topology of on bounded subsets of . For with and , we define , then , In particular, if is -bounded and under the norm in , then weakly in , strongly in , and weakly in . Let be a -bounded open convex subset and let be a fixed point. Let be a -continuous map from onto satisfying the following.

Set , , where denotes the -boundary of . For we introduce the class of mappings with the following properties.

The following is a variant weak linking theorem in [22].

Theorem 3.1. Let the family of -functionals have the form Assume that the following conditions hold. (1).(2) or as .(3) is -upper semicontinuous; is weakly sequentially continuous on . Moreover, maps bounded sets into bounded sets.(4).Then for almost all , there exists a sequence such that where .

Remark 3.2. Consider defined as in Section 2. Obviously, is reflexive. For with . Let and , then . It is easy to see that is a closed and separable subspace of . For any can be written as with and . For , let , then for . Define as . Then it is easy to check that satisfy (i), (ii), and (iii). If we let and , then links (see Lemmas 4.2 and 4.3 in Section 4).

4. The Proof of the Main Results

Consider the functional

for . Then by assumptions and Lemma 2.1, . A standard argument shows that any critical point of is a homoclinic orbit of (1.1) (cf. [14]).

Set

Then and . By and ,

where, as below, stands for some generic positive constant.

Together with , one has

Thus or if .

Lemma 4.1. is -upper semicontinuous and is weakly sequentially continuous.

Proof. For any , assume that with . Let with and . Then in and hence . Since and we have .
By (4.4),
Then . By Lemma 2.1, in in and a.e. for . By Fatou's Lemma, . Therefore, is -upper semicontinuous.
Let in , then in . By (4.3), in and for any . Therefore, .

Lemma 4.2. There exist , such that , with , .

Proof. By (4.4) and Lemma 2.1, The conclusion is obvious.

Lemma 4.3. There exists such that and for all , where and , .

Proof. For by (4.4),
Since is continuously embedded in for there exists a continuous projection from in to . Thus, for some and then
and thus the lemma follows easily.

Combining Lemmas 4.1–4.3 and Theorem 3.1, we get the following.

Lemma 4.4. Under and , for almost every , there exist such that

We need the following lemma which is a special case of a more general result due to Lions [27, 28].

Lemma 4.5. Let and . If is bounded and if where , then in for .

Lemma 4.6. Under , let be fixed. For the sequence in Lemma 4.4, there exist such that, up to a subsequence, satisfies , and

Proof. Write with and . Since , , let denote the part of in . Then by , Obviously, has a bounded inverse . Since then we have Set , then We claim that is nonvanishing, that is, there exist , and such that Indeed, if not, by (4.14), is bounded in . Lemma 4.5 shows that in for . By (4.3), Hence Thus, Therefore, for any , Thus we obtain and then a contradiction. Choose such that and let In view of the invariance of under the action , . Since commutes with , then also does. Therefore . By (4.15) Clearly, Thus, up to a subsequence, we assume that We now establish that If not, in and then which implies that contradicting (4.22). By Lemma 4.1, is weakly continuous, hence we have By Fatou's Lemma, we obtain

As a straightforward consequence of Lemmas 4.4 and 4.6, we have the following.