𝐽-Self-Adjoint Extensions for a Class of Discrete Linear Hamiltonian Systems

Abstract

This paper is concerned with formally 𝐽-self-adjoint discrete linear Hamiltonian systems on finite or infinite intervals. The minimal and maximal subspaces are characterized, and the defect indices of the minimal subspaces are discussed. All the 𝐽-self-adjoint subspace extensions of the minimal subspace are completely characterized in terms of the square summable solutions and boundary conditions. As a consequence, characterizations of all the 𝐽-self-adjoint subspace extensions are given in the limit point and limit circle cases.

1. Introduction

Consider the following discrete linear Hamiltonian system: 𝒥Δ𝑦(𝑡)=(𝑃(𝑡)+𝜆𝑊(𝑡))𝑅𝑦(𝑡),𝑡∈ℐ,1_𝜆 where ℐ∶={𝑡}^𝑏_𝑡=𝑎, 𝑎 is a finite integer or 𝑎=−∞, 𝑏 is a finite integer or 𝑏=+∞, and 𝑏−𝑎≥1; Δ is the forward difference operator, that is, Δ𝑦(𝑡)=𝑦(𝑡+1)−𝑦(𝑡); 𝒥 is the canonical symplectic matrix, that is, 𝒥=0−𝐼_𝑛𝐼_𝑛0,(1) where 𝐼_𝑛 is the 𝑛×𝑛 unit matrix; the weighted function 𝑊(𝑡) is a 2𝑛×2𝑛 real symmetric matrix with 𝑊(𝑡)≥0 for 𝑡∈ℐ, and it is of the block diagonal form, 𝑊(𝑡)=diag𝑊₁(𝑡),𝑊₂(𝑡),(2) where 𝑃(𝑡) is a 2𝑛×2𝑛 complex symmetric matrix, that is, 𝑃^𝑇(𝑡)=𝑃(𝑡). The partial right shift operator 𝑅(𝑦)(𝑡)=(𝑢^𝑇(𝑡+1),𝑣^𝑇(𝑡))^𝑇 with 𝑦(𝑡)=(𝑢^𝑇(𝑡),𝑣^𝑇(𝑡))^𝑇 and 𝑢(𝑡),𝑣(𝑡)∈ℂ^𝑛; 𝜆 is a complex spectral parameter.

For briefness, denote ℐ=[𝑎,𝑏] in the case where 𝑎 and 𝑏 are finite integers; ℐ=[𝑎,+∞) in the case where 𝑎 is finite and 𝑏=+∞; ℐ=(−∞,𝑏] in the case where 𝑎=−∞ and 𝑏 is finite; ℐ=(−∞,+∞) in the case where 𝑎=−∞ and 𝑏=+∞.

Since 𝑃(𝑡) is symmetric, it can be blocked as 𝑃(𝑡)=−𝐶(𝑡)𝐴^𝑇(𝑡)𝐴(𝑡)𝐵(𝑡),(3) where 𝐴, 𝐵, and 𝐶 are 𝑛×𝑛 complex-valued matrices with 𝐶^𝑇=𝐶 and 𝐵^𝑇=𝐵. Then, 1_𝜆 can be rewritten as 𝐼_𝑛−𝐴(𝑡)𝑢(𝑡+1)=𝑢(𝑡)+𝐵(𝑡)+𝜆𝑊₂(𝑡)𝑣(𝑡),𝑣(𝑡+1)=𝐶(𝑡)−𝜆𝑊₁(𝑡)𝑢()+𝐼𝑡+1_𝑛−𝐴^𝑇(𝑡)𝑣(𝑡)2,𝑡∈ℐ.^𝜆

To ensure the existence and uniqueness of the solution of any initial value problem for 1_𝜆, we always assume in the present paper that (𝐴₁)𝐼_𝑛−𝐴(𝑡) is invertible in ℐ.

It can be easily verified that 1_𝜆 contains the following complex coefficients vector difference equation of order 2𝑚: ^𝑚_𝑗=0(−1)^𝑗Δ^𝑗𝑝_𝑗(𝑡)Δ^𝑗𝑧𝑡−𝑗=𝜆𝑤(𝑡)𝑧(𝑡),𝑡∈ℐ,3_𝜆 where 𝑝_𝑗(𝑡) are 𝑙×𝑙 complex-valued matrices with 𝑝^𝑇_𝑗(𝑡)=𝑝_𝑗(𝑡), 0≤𝑗≤𝑚; 𝑝_𝑚(𝑡) is invertible in ℐ; 𝑤(𝑡) is an 𝑙×𝑙 real-valued with 𝑤(𝑡)≥0. In fact, by letting 𝑦=(𝑢^𝑇,𝑣^𝑇)^𝑇 with 𝑢=(𝑢^𝑇₁,𝑢^𝑇₂,…,𝑢^𝑇_𝑚)^𝑇, 𝑣=(𝑣^𝑇₁,𝑣^𝑇₂,…,𝑣^𝑇_𝑚)^𝑇, and 𝑢_𝑗(𝑡)=Δ^𝑗−1𝑧,𝑣𝑡−𝑗_𝑗(𝑡)=^𝑚_𝑘=𝑗(−1)^𝑘−𝑗Δ^𝑘−𝑗𝑝_𝑘(𝑡)Δ^𝑘𝑧(𝑡−𝑘)(4) for 1≤𝑗≤𝑚, 3_𝜆 can be converted into 1_𝜆, as well as 2_𝜆, with −𝐶(𝑡)=diag𝑝₀(𝑡),𝑝₁(𝑡),…,𝑝_𝑚−1(𝑡,𝐴(𝑡)=)0𝐼_{𝑙(𝑚−1)},𝐵(𝑡)00=diag0,…,0,𝑝_𝑚⁻¹(𝑡,𝑊(𝑡){𝑤(𝑡)}.)=diag,0,…,0(5) It is obvious that (𝐴₁) is satisfied for 3_𝜆.

The spectral theory of self-adjoint operators and self-adjoint extensions of symmetric operators (i.e., densely defined Hermitian operators) in Hilbert spaces has been well developed (cf. [1–4]). In general, under certain definiteness conditions, a formally self-adjoint differential expression can generate a minimal operator which is symmetric, and the defect index of the minimal operator is equal to the number of linearly independent square integrable solutions. All the characterizations of self-adjoint extensions of differential equation are obtained [5–8].

However, for difference equations, it was found in [9] that the minimal operator defined in [10] may be neither densely defined nor single-valued even if the definiteness condition is satisfied. This is an important difference between the differential and difference equations. In order to study the self-adjoint extensions of nondensely defined or multivalued Hermitian operators, some scholars tried to extend the concepts and theory for densely defined Hermitian operators to Hermitian subspaces [11–15]. Recently, Shi extended the Glazman-Krein-Naimark (GKN) theory for symmetric operators to Hermitian subspaces [9]. Applying this GKN theory, the first author, with Shi and Sun, gave complete characterizations of self-adjoint extensions for second-order formally self-adjoint difference equations and general linear discrete Hamiltonian systems, separately [16, 17].

We note that when the coefficient 𝑃(𝑡) in 1_𝜆 is not a Hermitian matrix, that is, 𝑃^∗(𝑡)≠𝑃(𝑡), system 1_𝜆 is not formally self-adjoint, and the minimal subspace generated by 1_𝜆 is not Hermitian. Hence the spectral theory of self-adjoint operators or self-adjoint subspaces is not applicable. To solve this problem, Glazman introduced a concept of 𝐽-symmetric operators in [3, 18] where 𝐽 is an operator. The minimal operators generated by certain differential expressions are 𝐽-symmetric operators in the related Hilbert spaces [19, 20]. Monaquel and Schmidt [21] discussed the 𝑀-functions of the following discrete Hamiltonian system: 𝒥Δ𝑧₁(𝑡)∇𝑧₂(𝑡)=(𝑄(𝑡)(𝑡𝑧+𝜆𝐻))¹(𝑡)𝑧₂(𝑡),𝑡∈[0,+∞),4_𝜆 where ∇ is the backward difference operator, that is, ∇𝑧(𝑡)=𝑧(𝑡)−𝑧(𝑡−1), and weighted function 𝐻(𝑡)=diag{𝐻₁(𝑡),𝐻₂(𝑡)}. By letting 𝑧₁(𝑡)=𝑣(𝑡), 𝑧₂(𝑡)=𝑢(𝑡+1), 1_𝜆 can be converted into (4_−𝜆) with 𝑄(𝑡)=−𝐵(𝑡)−𝐴(𝑡)−𝐴^𝑇(𝑡)𝐶(𝑡),𝐻(𝑡)=𝑊₂(𝑡)00𝑊₁(𝑡).(6)

In [22], the result that every 𝐽-Hermitian subspace has a 𝐽-self-adjoint subspace extension has been given. Furthermore, a result about 𝐽-self-adjoint subspace extension was obtained [22], which can be regarded as a GKN theorem for 𝐽-Hermitian subspaces.

In the present paper, enlightened by the methods used in the study of self-adjoint subspace extensions of Hermitian subspaces, we will study the 𝐽-self-adjoint subspace extensions of the minimal operator corresponding to system 1_𝜆. A complete characterization of them in terms of boundary conditions is given by employing the GKN theorem for 𝐽-Hermitian subspaces. The rest of this paper is organized as follows. In Section 2, some basic concepts and useful results about subspaces are briefly recalled. In Section 3, a conjugation operator 𝐽 is defined in the corresponding Hilbert space, and the maximal and minimal subspaces are discussed. In Section 4, the description of the minimal subspaces is given by the properties of their elements at the endpoints of the discussed intervals, the defect indices of minimal subspaces are discussed, and characterizations of the maximal subspaces are established. Section 5 pays attention to two characterizations of all the self-adjoint subspace extensions of the minimal subspace in terms of boundary conditions via linearly independent square summable solutions of 1_𝜆. As a consequence, characterizations of all the self-adjoint subspace extensions are given in two special cases: the limit point and limit circle cases.

2. Fundamental Results on Subspaces

In this section, we recall some basic concepts and useful results about subspaces. For more results about nondensely defined 𝐽-Hermitian operators or 𝐽-Hermitian subspaces, we refer to [17–19, 22] and some references cited therein. In addition, some properties of solutions of 1_𝜆 and a result about matrices are given at the end of this section.

By ℝ and ℂ we denote the sets of the real and the complex numbers, respectively. Let 𝑋 be a complex Hilbert space equipped with inner product ⟨⋅,⋅⟩, 𝑇 and 𝑆 two linear subspaces (briefly, subspace) in 𝑋²∶=𝑋×𝑋, and 𝜆∈ℂ. Denote Dom𝑇∶=𝑥∈𝑋∶𝑥,𝑓∈𝑇forsome𝑓∈𝑋,Ran𝑇∶=𝑓∈𝑋∶𝑥,𝑓∈𝑇forsome,{()},𝑇𝑥∈𝑋Ker𝑇∶=𝑥∈𝑋∶𝑥,0∈𝑇_∗∶=𝑥,𝑓∈𝑋²∶=∀,∶.𝑥,𝑔𝑓,𝑦𝑦,𝑔∈𝑇𝑇−𝜆𝐼∶=𝑥,𝑓−𝜆𝑥𝑥,𝑓∈𝑇(7) If 𝑇∩𝑆={0}, we write 𝑇∔𝑆∶=𝑥+𝑦,𝑓+𝑔∶𝑥,𝑓∈𝑇,𝑦,𝑔∈𝑆,(8) which is denoted by 𝑇⊕𝑆 in the case that 𝑇 and 𝑆 are orthogonal.

Denote 𝑇(𝑥)∶=𝑓∈𝑋∶𝑥,𝑓∈𝑇.(9) It can be easily verified that 𝑇(0)={0} if and only if 𝑇 can determine a unique linear operator from Dom𝑇 into 𝑋 whose graph is just 𝑇. For convenience, we will identify a linear operator in 𝑋 with a subspace in 𝑋² via its graph.

Definition 1 (see [11]). Let 𝑇 be a subspace in 𝑋². (1)𝑇 is said to be a Hermitian subspace if 𝑇⊂𝑇^∗. Furthermore, 𝑇 is said to be a Hermitian operator if it is an operator, that is, 𝑇(0)={0}. (2)𝑇 is said to be a self-adjoint subspace if 𝑇=𝑇^∗. Furthermore, 𝑇 is said to be a self-adjoint operator if it is an operator, that is, 𝑇(0)={0}. (3) Let 𝑇 be a Hermitian subspace. 𝑇₁ is said to be a self-adjoint subspace extension (briefly, SSE) of 𝑇 if 𝑇⊂𝑇₁ and 𝑇₁ is a self-adjoint subspace. (4) Let 𝑇 be a Hermitian operator. 𝑇₁ is said to be a self-adjoint operator extension (briefly, SOE) of 𝑇 if 𝑇⊂𝑇₁ and 𝑇₁ is a self-adjoint operator.

Lemma 2 (see [11]). Let 𝑇 be a subspace in 𝑋². Then (1)𝑇^∗ is a closed subspace in 𝑋²; (2)𝑇^∗=(𝑇)^∗ and 𝑇^∗∗=𝑇, where 𝑇 is the closure of 𝑇; (3)Ker𝑇^∗=(Ran𝑇)^⟂=(Ran𝑇)^⟂.

In [19], an operator 𝐽 defined in 𝑋 is said to be a conjugation operator if for all 𝑥,𝑦∈𝑋, 𝐽𝑥,𝐽𝑦=𝑦,𝑥,𝐽²𝑥=𝑥.(10)

Definition 3. Let 𝑇 be a subspace in 𝑋² and 𝐽 be a conjugation operator. (1)The 𝐽-adjoint of 𝑇 is defined by 𝑇^∗_𝐽∶=𝑦,𝑔∈𝑋²∶𝑓,𝐽𝑦=𝑥,𝐽𝑔∀𝑥,𝑓∈𝑇.(11)(2)𝑇 is said to be a 𝐽-Hermitian subspace if 𝑇⊂𝑇^∗_𝐽. Furthermore, 𝑇 is said to be a 𝐽-Hermitian operator if it is an operator, that is, 𝑇(0)={0}. (3)𝑇 is said to be a 𝐽-self-adjoint subspace if 𝑇=𝑇^∗_𝐽. Furthermore, 𝑇 is said to be a 𝐽-self-adjoint operator if it is an operator, that is, 𝑇(0)={0}. (4)Let 𝑇 be a 𝐽-Hermitian subspace. 𝑇₁ is said to be a 𝐽-self-adjoint subspace extension (briefly, 𝐽-SSE) of 𝑇 if 𝑇⊂𝑇₁ and 𝑇₁ is a 𝐽-self-adjoint subspace. (5)Let 𝑇 be a 𝐽-Hermitian operator. 𝑇₁ is said to be a 𝐽-self-adjoint operator extension (briefly, 𝐽-SOE) of 𝑇 if 𝑇⊂𝑇₁ and 𝑇₁ is a 𝐽-self-adjoint operator.

Remark 4. (1) It can be easily verified that 𝑇^∗_𝐽 is a closed subspace. Consequently, a 𝐽-self-adjoint subspace 𝑇 is a closed subspace since 𝑇=𝑇^∗_𝐽. In addition, 𝑆^∗_𝐽⊂𝑇^∗_𝐽 if 𝑇⊂𝑆.
(2) From the definition, we have that ⟨𝑓,𝐽𝑦⟩=⟨𝑥,𝐽𝑔⟩ holds for all (𝑥,𝑓)∈𝑇 and (𝑦,𝑔)∈𝑇^∗_𝐽, and that 𝑇 is a 𝐽-Hermitian subspace if and only if ⟨𝑓,𝐽𝑦⟩=⟨𝑥,𝐽𝑔⟩ for all (𝑥,𝑓),(𝑦,𝑔)∈𝑇.

Lemma 5 (see [22]). Let 𝑇 be a subspace in 𝑋². Then (1)𝑇^∗∶={(𝐽𝑦,𝐽𝑔)∶(𝑦,𝑔)∈𝑇^∗_𝐽}; (2)𝑇^∗_𝐽∶={(𝐽𝑦,𝐽𝑔)∶(𝑦,𝑔)∈𝑇^∗}.

It follows from Lemmas 2 and 5 that 𝑇^∗_𝐽=(𝑇)^∗_𝐽, and 𝑇 is 𝐽-Hermitian if 𝑇 is 𝐽-Hermitian.

Lemma 6 (see [22]). Every J-Hermitian subspace has a 𝐽-SSE.

Definition 7. Let T be a 𝐽-Hermitian subspace. Then 𝑑(𝑇)=(1/2)dim𝑇^∗_𝐽/𝑇 is called to be the defect index of 𝑇.

Next, we introduce a form on 𝑋²×𝑋² by ∶−,,𝑥,𝑓𝑦,𝑔∶=𝑓,𝐽𝑦𝑥,𝐽𝑔𝑥,𝑓𝑦,𝑔∈𝑋².(12)

Lemma 8 (see [22]). Let 𝑇 be a 𝐽-Hermitian subspace. Then 𝑇=𝑥,𝑓∈𝑇^∗_𝐽∶𝑥,𝑓∶𝑦,𝑔=0,∀𝑦,𝑔∈𝑇^∗_𝐽.(13)

Lemma 9 (see [22]). Let 𝑇 be a closed 𝐽-Hermitian subspace in 𝑋² and satisfy 𝑑=𝑑(𝑇)<+∞. Then a subspace 𝑇₁ is a 𝐽-SSE of 𝑇 if and only if 𝑇⊂𝑇₁⊂𝑇^∗_𝐽 and there exists {(𝑥_𝑗,𝑓_𝑗)}^𝑑_𝑗=1⊂𝑇^∗_𝐽 such that (1)(𝑥₁,𝑓₁),(𝑥₂,𝑓₂),…,(𝑥_𝑑,𝑓_𝑑) are linearly independent in 𝑇^∗_𝐽 (modulo 𝑇); (2)[(𝑥_𝑗,𝑓_𝑗)∶(𝑥_𝑘,𝑓_𝑘)]=0, 1≤𝑗, 𝑘≤𝑑;(3)𝑇₁={(𝑦,𝑔)∈𝑇^∗_𝐽∶[(𝑦,𝑔)∶(𝑥_𝑗,𝑓_𝑗)]=0,1≤𝑗≤𝑑}.

Lemma 9 can be regarded as a GKN theorem for 𝐽-Hermitian subspaces. A set of {(𝑥_𝑗,𝑓_𝑗)}^𝑑_𝑗=1⊂𝑇^∗_𝐽 which is satisfying (1) and (2) in Lemma 9 is called a GKN set of 𝑇.

Definition 10. Let 𝑇 be a subspace in 𝑋².(1) The set 𝜌(𝑇)∶=𝜆∈ℂ∶(𝑇−𝜆)⁻¹isaboundedlinearoperatordefinedin𝑋(14) is called the resolvent set of 𝑇.(2) The set 𝜎(𝑇)∶=ℂ⧵𝜌(𝑇) is called the spectrum of 𝑇. (3) The set Γ(𝑇)∶=𝜆∈ℂ∶∃𝑐(𝜆)>0s.t.𝑓−𝜆𝑥≥𝑐(𝜆)‖𝑥‖,∀𝑥,𝑓∈𝑇(15) is called to be the regularity field of 𝑇.
It is evident that 𝜌(𝑇)⊂Γ(𝑇) for any subspace 𝑇 in 𝑋².

Lemma 11 (see [22]). Let 𝑇 be a 𝐽-Hermitian subspace in 𝑋² with Γ(𝑇)≠∅, and 𝜆∈Γ(𝑇). Then 𝑇^∗_𝐽=𝑇∔𝑦,𝑔∈𝑇^∗_𝐽∶𝑔−𝜆𝑦∈Ker𝑇^∗−𝜆,𝑑(𝑇)=dimRan(𝑇−𝜆)^⟂=dimKer𝑇^∗−𝜆.(16)

The following is a well-known result on the rank of matrices.

Lemma 12. Let 𝐴 be an 𝑚×𝑙 matrix and 𝐵 an 𝑙×𝑛 matrix. Then (){}.rank𝐴+rank𝐵−𝑙≤rank𝐴𝐵≤minrank𝐴,rank𝐵(17) In particular, if 𝐴𝐵=0, then rank𝐴+rank𝐵≤𝑙.(18)

3. Relationship between the Maximal and Minimal Subspaces

This section is divided into three subsections. In the first subsection, we define a conjugation operator in a Hilbert space. In the second subsection, we define maximal and minimal subspaces generated by 1_𝜆 and discuss relationship between them. In the last subsection, we discuss the definiteness condition corresponding to 1_𝜆.

3.1. Conjugation Operator

In this subsection, we define a conjugation operator in a Hilbert space and then discuss its properties.

Since 𝑏 and 𝑎 may be finite or infinite, we introduce the following conventions for briefness: 𝑏+1 means +∞ in the case of 𝑏=+∞ and 𝑎−1 means −∞ in the case of 𝑎=−∞. Denote ℐ^∗∶={𝑡}^𝑏+1_𝑡=𝑎,𝑙(ℐ)∶=𝑦∶𝑦=𝑦(𝑡)_{𝑡∈ℐ∗}⊂ℂ^2𝑛.(19) For any 2𝑛×2𝑛 Hermitian matrix 𝑊(𝑡)≥0 defined in ℐ, we define ℒ²_𝑊(ℐ)∶=𝑦∈𝑙(ℐ)∶_𝑡∈ℐ𝑅𝑦^∗(𝑡)𝑊(𝑡)𝑅𝑦(𝑡)<+∞(20) with the semiscalar product 𝑦,𝑧∶=_𝑡∈ℐ𝑅^∗(𝑧)(𝑡)𝑊(𝑡)𝑅𝑦(𝑡).(21) Furthermore, denote ‖𝑦‖∶=(⟨𝑦,𝑦⟩)^1/2 for 𝑦∈ℒ²_𝑊(ℐ). Since the weighted function 𝑊(𝑡) may be singular in ℐ, ‖⋅‖ is a seminorm. Introduce the quotient space 𝐿²_𝑊(ℐ)ℒ∶=^2𝑊(ℐ)𝑦∈ℒ²_𝑊(ℐ)∶𝑦.=0(22) Then 𝐿²_𝑊(ℐ) is a Hilbert space with the inner product ⟨⋅,⋅⟩.

For a function 𝑦∈ℒ²_𝑊(ℐ), denote by 𝑦^𝜋 the corresponding class in 𝐿²_𝑊(ℐ). And for any 𝑦^𝜋∈𝐿²_𝑊(ℐ), denote by 𝑦∈ℒ²_𝑊(ℐ) a representative of 𝑦^𝜋. It is evident that ⟨𝑦^𝜋,𝑧^𝜋⟩=⟨𝑦,𝑧⟩ for any 𝑦^𝜋,𝑧^𝜋∈𝐿²_𝑊(ℐ).

For any 𝑦∈ℒ²_𝑊(ℐ), denote by 𝐽₀𝑦 the conjugation of 𝑦; that is, 𝐽₀𝑦∶=𝑦(𝑡)_{𝑡∈ℐ∗},𝑦∈ℒ²_𝑊(ℐ).(23) It can be easily verified that 𝑦∈ℒ²_𝑊(ℐ) if and only if 𝐽₀𝑦∈ℒ²_𝑊(ℐ). Here 𝑊 is the conjugation of matrix 𝑊. Since each 𝑦^𝜋∈𝐿²_𝑊(ℐ) is an equivalent class, we define a operator 𝐽 defined on 𝐿²_𝑊(ℐ) by 𝐽𝑦^𝜋∶=𝐽₀𝑦^𝜋,𝑦^𝜋∈𝐿²_𝑊(ℐ).(24) The following result is obtained.

Lemma 13. 𝐽 defined by (24) is a conjugation operator defined on 𝐿²_𝑊(ℐ) if and only if 𝑊(𝑡) is real and symmetric in ℐ.

Proof. The sufficiency is evident. Next, we consider the necessity. Assume that 𝐽 defined by (24) is a conjugation operator in 𝐿²_𝑊(ℐ). Then for any 𝑥^𝜋,𝑦^𝜋∈𝐿²_𝑊(ℐ), it follows from ⟨𝐽𝑥^𝜋,𝐽𝑦^𝜋⟩=⟨𝑦^𝜋,𝑥^𝜋⟩ that _𝑡∈ℐ𝑅(𝑥)^∗(𝑡)𝑊^𝑇(𝑡)𝑅𝑦(𝑡)=_𝑡∈ℐ𝑅(𝑥)^∗(𝑡)𝑊(𝑡)𝑅𝑦(𝑡).(25) By the arbitrariness of 𝑥^𝜋,𝑦^𝜋, one has that 𝑊(𝑡)=𝑊^𝑇(𝑡). This, together with 𝑊(𝑡)=𝑊^∗(𝑡), yields that 𝑊(𝑡) is real. The proof is complete.

For any 𝑥,𝑦∈𝑙(ℐ), we denote 𝑥,𝑦(𝑡)=𝑦^𝑇(𝑡)𝒥𝑥(𝑡),𝑡∈ℐ^∗,(26) where 𝒥 is the canonical symplectic matrix given in Section 1. In the case of 𝑏=+∞, if lim_𝑡→𝑏(𝑥,𝑦)(𝑡) exists and is finite, then its limit is denoted by (𝑥,𝑦)(+∞). In the case of 𝑎=−∞, if lim_𝑡→𝑎(𝑥,𝑦)(𝑡) exists and is finite, then its limit is denoted by (𝑥,𝑦)(−∞).

Denote 𝜏𝑦(𝑡)(𝑡)(𝑡)𝑅𝑦(𝑡),𝛿𝑦(𝑡)(𝑡)−∶=𝒥Δ𝑦−𝑃∶=𝒥Δ𝑦𝑃(𝑡)𝑅𝑦(𝑡),(27) where 𝜏 and 𝛿 are called the natural difference operators corresponding to system 1_𝜆. The following result can be easily verified, and so we omit the proof.

Lemma 14. Assume that (𝐴₁) holds. Let 𝑥,𝑦∈𝑙(ℐ). (1)𝛿(𝐽₀𝑦)=𝐽₀𝜏(𝑦),𝜏(𝐽₀𝑦)=𝐽₀𝛿(𝑦). (2) For any 𝑠,𝑘∈ℐ, ^𝑘_𝑡=𝑠𝑅𝐽₀𝑦^∗(𝑡)𝜏(𝑥𝑡)−𝐽)(₀𝜏𝑦^∗(𝑡)𝑅(𝑥𝑡=𝑡)()𝑥,𝑦()_𝑠^𝑘+1.(28)(3) For any 𝜆∈ℂ, 𝑐₀∈ℐ, and any two solutions 𝑥(𝑡) and 𝑦(𝑡) of 1_𝜆, it follows that 𝑥,𝑦(𝑡)=𝑥,𝑦𝑐₀,𝑡∈ℐ^∗.(29)
Moreover, let 𝑌(𝑡,𝜆) be a fundamental solution of 1_𝜆, then 𝑌^𝑇(𝑡,𝜆)𝒥𝑌(𝑡,𝜆)=𝑌^𝑇𝑐₀,𝜆𝒥𝑌𝑐₀,𝜆,𝑡∈ℐ^∗.(30)

3.2. Relationship between the Maximal and Minimal Subspaces

In this subsection, we first introduce the maximal and minimal subspaces corresponding to 1_𝜆 and then show that the minimal subspace is 𝐽-Hermitian, and its 𝐽-adjoint subspace is just the maximal subspace.

Denote ℒ²_𝑊,0(ℐ)∶=𝑦∈ℒ²_𝑊(ℐ)∶thereexisttwointegers𝑠,𝑘∈ℐwith𝑠≤𝑘suchthat𝑦(𝑡)=0for𝑡≤𝑠and𝑡≥𝑘+1,(31) and define 𝐻(𝜏)∶=𝑦^𝜋,𝑔^𝜋∈𝐿²_𝑊(ℐ)²∶∃𝑦∈𝑦^𝜋s.t.𝑦(𝑡)=𝑊(𝑡)𝑅𝑔(𝑡)inℐ,𝐻₀₀(𝜏)∶=𝑦^𝜋,𝑔^𝜋∈𝐻(𝜏)∶∃𝑦∈𝑦^𝜋s.t.𝑦∈ℒ²_𝑊,0(ℐ)and𝜏𝑦(𝑡)=𝑊(𝑡)𝑅𝑔(𝑡)inℐ.(32) It can be easily verified that 𝐻(𝜏) and 𝐻₀₀(𝜏) are both linear subspaces in (𝐿²_𝑊(ℐ))². Here, 𝐻(𝜏) and 𝐻₀₀(𝜏) are called the maximal and preminimal subspaces corresponding to 𝜏 or 1_𝜆 in (𝐿²_𝑊(ℐ))², and 𝐻₀(𝜏)∶=𝐻₀₀(𝜏) is called the minimal subspace corresponding to 𝜏 or 1_𝜆 in (𝐿²_𝑊(ℐ))².

Since the end points 𝑎 and 𝑏 may be finite or infinite, we need to divide ℐ into two subintervals in order to characterize the maximal and minimal subspaces in a unified form. Choose 𝑎<𝑐₀<𝑏 and fix it. Denote ℐ_𝑎∶={𝑡}^𝑐0−1_𝑡=𝑎,ℐ_𝑏∶={𝑡}^𝑏_𝑡=𝑐0,(33) and denote by ⟨⋅,⋅⟩_𝑎, ⟨⋅,⋅⟩_𝑏 and ‖⋅‖_𝑎, ‖⋅‖_𝑏 the inner products and norms of ℒ²_𝑊(ℐ_𝑎), ℒ²_𝑊(ℐ_𝑏), respectively. Let ℒ²_𝑊,0(ℐ_𝑎) and ℒ²_𝑊,0(ℐ_𝑏) be defined by (31) with ℐ replaced by ℐ_𝑎 and ℐ_𝑏, respectively. Furthermore, let 𝐻_𝑎(𝜏) and 𝐻_𝑎,00(𝜏) be the left maximal and preminimal subspaces defined by (32) with ℐ replaced by ℐ_𝑎, respectively, and 𝐻_𝑏(𝜏) and 𝐻_𝑏,00(𝜏) the right maximal and preminimal subspaces defined by (32) with ℐ replaced by ℐ_𝑏, respectively. The subspaces 𝐻_𝑎,0(𝜏)∶=𝐻_𝑎,00(𝜏) and 𝐻_𝑏,0(𝜏)∶=𝐻_𝑏,00(𝜏) are called the left and right minimal subspaces corresponding to system 1_𝜆 in ℐ_𝑎 and ℐ_𝑏, respectively. Similarly, we can define 𝐻(𝛿), 𝐻₀₀(𝛿), and 𝐻₀(𝛿); 𝐻_𝑏(𝛿), 𝐻_𝑏,00(𝛿), and 𝐻_𝑏,0(𝛿); 𝐻_𝑎(𝛿), 𝐻_𝑎,00(𝛿) and 𝐻_𝑎,0(𝛿).

The following result is directly derived from (1) of Lemma 14.

Lemma 15. Assume that (𝐴₁) holds. Then (𝑦^𝜋,𝑔^𝜋)∈𝐻(𝜏) if and only if (𝐽𝑦^𝜋,𝐽𝑔^𝜋)∈𝐻(𝛿).

In order to study properties of the above subspaces, we first make some preparation.

Let 𝑌(𝑡) be the fundamental solution matrix of (1₀) with 𝑌(𝑐₀)=𝐼_2𝑛. For any finite subinterval ℐ^′ with 𝑐₀∈ℐ^′⊂ℐ, denote Φ_ℐ′∶=_{𝑡∈ℐ′}𝑅(𝑌)^∗(𝑡)𝑊(𝑡)𝑅(𝑌)(𝑡).(34) It is evident that Φ_ℐ′ is a 2𝑛×2𝑛 positive semidefinite matrix and dependent on ℐ^′. By the same method used in [23, Lemma 3.2], it follows that there exists a finite subinterval ℐ₀ with 𝑐₀∈ℐ₀⊂ℐ such that rankΦ_ℐ0=rankΦ_ℐ′,RanΦ_ℐ0=RanΦ_ℐ′,(35) for any finite subinterval ℐ^′ with ℐ₀⊂ℐ^′⊂ℐ. In the present paper, we will always denote ℐ₀∶=[𝑠₀,𝑡₀] and define rankΦ_ℐ∶=rankΦ_ℐ0,RanΦ_ℐ∶=RanΦ_ℐ0,(36) whenever ℐ is finite or infinite. In the case that ℐ is finite, ℐ₀ can be taken as ℐ.

In the case that ℐ is finite, we define 𝜙_ℐ∶𝐿²_𝑊(ℐ)⟶ℂ^2𝑛,𝑔^𝜋⟼_𝑠∈ℐ𝑅𝐽₀𝑌^∗(𝑠)𝑊(𝑠)𝑅𝑔(𝑠).(37) It is evident that 𝜙_ℐ is a bounded linear map and its range is a closed subset in ℂ^2𝑛.

In the case that ℐ is infinite, that is, ℐ=[𝑎,+∞) or ℐ=(−∞,𝑏] or ℐ=(−∞,+∞), where 𝑎, 𝑏 are finite integers, we introduce the following subspaces of 𝐿²_𝑊(ℐ), respectively: 𝐿²_𝑊,1(ℐ)∶=𝑦^𝜋∈𝐿²_𝑊(ℐ)∶∃𝑘∈ℐs.t.𝑊(𝑡)𝑅𝑦(𝑡)=0for𝑡≥𝑘,(38)𝐿²_𝑊,1(ℐ)∶=𝑦^𝜋∈𝐿²_𝑊(ℐ)∶∃𝑠∈ℐs.t.𝑊(𝑡)𝑅𝑦(𝑡)=0for𝑡≤𝑠,(39)𝐿²_𝑊,1(ℐ)∶=𝑦^𝜋∈𝐿²_𝑊(ℐ)∶∃𝑠≤𝑘∈ℐs.t.𝑊(𝑡)𝑅𝑦(𝑡)=0for𝑡≤𝑠and𝑡≥𝑘.(40) It can be easily shown that 𝐿²_𝑊,1(ℐ) is dense in 𝐿²_𝑊(ℐ). In this case, we define 𝜙_ℐ∶𝐿²_𝑊,1(ℐ)⟶ℂ^2𝑛,𝑔^𝜋⟼_𝑠∈ℐ𝑅𝐽₀𝑌^∗(𝑠)𝑊(𝑠)𝑅𝑔(𝑠).(41) By the method used in [23, Lemma 3.3], one has the following properties of 𝜙_ℐ.

Lemma 16. Assume that (𝐴₁) holds.(1)Ran𝜙_ℐ=RanΦ_ℐ. (2) In the case that ℐ is finite, 𝐿²_𝑊(ℐ)=Ker𝜙_ℐ⊕𝐽₀(𝑌𝜉)^𝜋∶𝜉∈RanΦ_ℐ;(42) in the case that ℐ is infinite, let 𝑙=rankΦ_ℐ. Then there exist linearly independent elements ℎ^𝜋_𝑗∈𝐿²_𝑊,1(ℐ), 1≤𝑗≤𝑙, such that 𝐿²_𝑊,1(ℐ)=Ker𝜙_ℐ∔spanℎ^𝜋₁,ℎ^𝜋₂,…,ℎ^𝜋_𝑙.(43)(3)Ker𝜙_ℐ⊂Ran(𝐻₀₀(𝜏)).

The following is the main result of this section.

Theorem 17. Assume that (𝐴₁) holds. Then 𝐻(𝜏)=(𝐻₀₀(𝜏))^∗_𝐽=(𝐻₀(𝜏))^∗_𝐽, 𝐻_𝑏(𝜏)=(𝐻_𝑏,00(𝜏))^∗_𝐽=(𝐻_𝑏,0(𝜏))^∗_𝐽, and 𝐻_𝑎(𝜏)=(𝐻_𝑎,00(𝜏))^∗_𝐽=(𝐻_𝑎,0(𝜏))^∗_𝐽.

Proof. Since the method of the proofs is similar, we only show the first assertion. By (𝐻₀(𝜏))^∗_𝐽=(𝐻₀₀(𝜏))^∗_𝐽, it suffices to show 𝐻(𝜏)=(𝐻₀₀(𝜏))^∗_𝐽.
We first show that 𝐻(𝜏)⊂(𝐻₀₀(𝜏))^∗_𝐽. Let (𝑦^𝜋,𝑔^𝜋)∈𝐻(𝜏). Then for any (𝑥^𝜋,𝑓^𝜋)∈𝐻₀₀(𝜏), there exists 𝑥∈𝑥^𝜋 with 𝑥∈ℒ²_𝑊(ℐ) such that 𝜏(𝑥)(𝑡)=𝑊(𝑡)𝑅(𝑓)(𝑡) in ℐ. So, it follows from (2) of Lemma 14 that 𝑓^𝜋,𝐽𝑦^𝜋−𝑥^𝜋,𝐽𝑔^𝜋=_𝑡∈ℐ𝑅𝐽₀𝑦^∗(𝑡)𝑊(𝑡)𝑅𝑓(𝑡)−𝑅𝐽₀𝑔^∗(𝑡)𝑊(𝑡)𝑅(𝑥)(𝑡)=_𝑡∈ℐ𝑅𝐽₀𝑦^∗(𝑡)𝜏(𝑥𝑡)−𝐽)(₀𝜏𝑦^∗(𝑡)𝑅(𝑥𝑡=(𝑡)()𝑥,𝑦)_𝑎^𝑏+1=0.(44) This implies that 𝐻(𝜏)⊂(𝐻₀₀(𝜏))^∗_𝐽.
Next, we show (𝐻₀₀(𝜏))^∗_𝐽⊂𝐻(𝜏). Fix any (𝑦^𝜋,𝑔^𝜋)∈(𝐻₀₀(𝜏))^∗_𝐽. It suffices to show that there exists 𝑦₀∈𝑦^𝜋 such that 𝜏(𝑦₀)(𝑡)=𝑊(𝑡)𝑅(𝑔)(𝑡) in ℐ. Let 𝑧 be a solution of 𝜏(𝑧)(𝑡)=𝑊(𝑡)𝑅(𝑔)(𝑡) on ℐ. For any (𝑥^𝜋,𝑓^𝜋)∈𝐻₀₀(𝜏), there exits 𝑥∈𝑥^𝜋 with 𝑥∈ℒ²_𝑊,0(ℐ) such that 𝜏(𝑥)(𝑡)=𝑊(𝑡)𝑅(𝑓)(𝑡) in ℐ. Thus, it follows from (2) of Lemma 14 that _𝑡∈ℐ𝑅𝐽₀𝑧^∗(𝑡)𝑊(𝑡)𝑅𝑓(𝑡)−𝑅𝐽₀𝑔^∗(𝑡)𝑊(𝑡)𝑅(𝑥𝑡=)()_𝑡∈ℐ𝑅𝐽₀𝑧^∗(𝑡)𝜏(𝑥)(𝑡)−𝐽₀𝜏(𝑧)^∗(𝑡)𝑅(𝑥𝑡=(𝑡)()𝑥,𝑧)()|_𝑎^𝑏+1=0.(45) In addition, it is clear that 𝑓^𝜋,𝐽𝑦^𝜋−𝑥^𝜋,𝐽𝑔^𝜋=_𝑡∈ℐ𝑅𝐽₀𝑦^∗(𝑡)𝑊(𝑡)𝑅𝑓(𝑡)−𝑅𝐽₀𝑔^∗(𝑡)𝑊(𝑡)𝑅(𝑥)(𝑡)=0.(46) Combining (45) and (46), one has that for all (𝑥^𝜋,𝑓^𝜋)∈𝐻₀₀(𝜏), _𝑡∈ℐ𝑅𝐽₀𝑦−𝑧^∗(𝑡)𝑊(𝑡)𝑅𝑓(𝑡)=0.(47) By (2) and (3) of Lemma 16, we get that for any ℎ^𝜋∈Ker𝜙_ℐ and any 𝜉∈ℂ^2𝑛_𝑡∈ℐ𝑅𝐽₀𝑦−𝑧−𝑌𝜉^∗(𝑡)𝑊(𝑡)𝑅(ℎ)(𝑡)=0.(48)
The following discussion is divided into two parts.
Case 1. ℐ is finite. It is evident that 𝑦^𝜋,𝑧^𝜋∈𝐿²_𝑊(ℐ). Then, from (2) of Lemma 16, there exists 𝜉₀∈RanΦ_ℐ such that (𝐽₀(𝑦−𝑧−𝑌𝜉₀))^𝜋∈Ker𝜙_ℐ. This, together with (48), implies that (𝐽₀(𝑦−𝑧−𝑌𝜉₀))^𝜋=0. This is equivalent to (𝑦−𝑧−𝑌𝜉₀)^𝜋=0. Let 𝑦₀(𝑡)∶=𝑧(𝑡)+𝑌(𝑡)𝜉₀. Then 𝑦₀∈𝑦^𝜋 and satisfies 𝜏𝑦₀(𝑡)=𝜏(𝑧)(𝑡)+𝜏(𝑌𝜉)(𝑡)=𝑊(𝑡)𝑅𝑔(𝑡),𝑡∈ℐ.(49) Hence, (𝑦^𝜋,𝑔^𝜋)∈𝐻(𝜏). Since (𝑦^𝜋,𝑔^𝜋)∈𝐻^∗₀₀(𝜏) is arbitrary, we have (𝐻₀₀(𝜏))^∗_𝐽⊂𝐻(𝜏).
Case 2. ℐ is infinite. We only consider the case that ℐ=[𝑎,+∞). For the other two cases, it can be proved with similar arguments.
Let rankΦ_ℐ=𝑙. With a similar argument as Case 2 of the proof of [23, Theorem 3.1], it can be shown that there exist linearly independent elements ℎ^𝜋_𝑗∈𝐿²_𝑊,1,1≤𝑗≤𝑙, and 𝜉₀∈RanΦ_ℐ such that (𝑦−𝑧−𝑌𝜉₀)^𝜋∈Ker𝜙_ℐ, 𝐿²_𝑊,1(ℐ)=Ker𝜙_ℐ∔spanℎ^𝜋₁,ℎ^𝜋₂,…,ℎ^𝜋_𝑙,_𝑡∈ℐ𝑅𝐽₀𝑦−𝑧−𝑌𝜉₀^∗(𝑡)𝑊(𝑡)𝑅ℎ_𝑗(𝑡)=0,1≤𝑗≤𝑙.(50) Combining (48)–(50), one has that for any ℎ^𝜋∈𝐿²_𝑊,1(ℐ)_𝑡∈ℐ𝑅𝐽₀𝑦−𝑧−𝑌𝜉₀^∗(𝑡)𝑊(𝑡)𝑅(ℎ)(𝑡)=0.(51) This implies that (𝑦−𝑧−𝑌𝜉₀)^𝜋=0 and consequently 𝑦₀∶=𝑧+𝑌𝜉₀ is a representative of 𝑦^𝜋 such that 𝜏(𝑦₀)(𝑡)=𝑊(𝑡)𝑅(𝑔)(𝑡). So (𝑦^𝜋,𝑔^𝜋)∈𝐻(𝜏). By the arbitrariness of (𝑦^𝜋,𝑔^𝜋) one has (𝐻₀₀(𝜏))^∗_𝐽⊂𝐻(𝜏).
The entire proof is complete.

The following result is directly derived from Lemmas 5 and 15, and Theorem 17.

Theorem 18. Assume that (𝐴₁) holds. Then 𝐻(𝛿)=(𝐻₀₀(𝜏))^∗, 𝐻_𝑏(𝛿)=(𝐻_𝑏,00(𝜏))^∗, and 𝐻_𝑎(𝛿)=(𝐻_𝑎,00(𝜏))^∗.

3.3. Definiteness Condition

In this subsection, we introduce the definiteness condition for 1_𝜆, and give some important results on it. Since the proofs are similar to those given in [23], we omit the proofs.

The definiteness condition for 1_𝜆 or 𝐻(𝜏) is given by the following. (𝐴₂) There exists a finite subinterval ℐ₁⊂ℐ such that for any 𝜆∈ℂ and for any nontrivial solution 𝑦(𝑡) of 1_𝜆, the following always holds: _𝑡∈ℐ1𝑅𝑦^∗(𝑡)𝑊(𝑡)𝑅𝑦(𝑡)>0.(52)In particular, the definiteness condition for 3_𝜆 can be described as there exists a finite subinterval ℐ₁⊂ℐ such that for any 𝜆∈ℂ and for any nontrivial solution 𝑧(𝑡) of 3_𝜆, the following always holds: _𝑡∈ℐ1𝑧^∗(𝑡)𝑤(𝑡)𝑧(𝑡)>0.(53)

Lemma 19. Assume that (𝐴₁) holds. Then (𝐴₂) holds if and only if there exists a finite subinterval ℐ₁⊂ℐ such that one of the following holds: (1)rankΦ_ℐ1=2𝑛; (2) for some 𝜆∈ℂ, every nontrivial solution 𝑦(𝑡) of 1_𝜆 satisfies _𝑡∈ℐ1𝑅𝑦^∗(𝑡)𝑊(𝑡)𝑅𝑦(𝑡)>0.(54)

By Lemma 19, if (52) (or (53)) holds for some 𝜆∈ℂ, then it holds for every 𝜆∈ℂ. In addition, if (𝐴₂) holds on some finite interval ℐ₁, then it holds on ℐ₀=[𝑠₀,𝑡₀].

The following is another sufficient and necessary condition for the definiteness condition.

Lemma 20. Assume that (𝐴₁) holds. Then (𝐴₂) holds if and only if for any (𝑦^𝜋,𝑔^𝜋)∈𝐻(𝜏), there exists a unique 𝑦∈𝑦^𝜋 such that 𝜏(𝑦)(𝑡)=𝑊(𝑡)𝑅(𝑔)(𝑡) for 𝑡∈ℐ.

Remark 21. (1) It can be easily verified that the definiteness condition for 𝐻(𝜏) holds if and only if that for 𝐻(𝛿) holds.
(2) In the following of the present paper, we always assume that (𝐴₂) holds. In this case, we can write (𝑦,𝑔^𝜋)∈𝐻(𝜏) instead of (𝑦^𝜋,𝑔^𝜋)∈𝐻(𝜏) in the rest of the present paper.
(3) Denote by (𝐴_𝑎,2) and (𝐴_𝑏,2), the definiteness conditions for 1_𝜆 in ℐ_𝑎 and ℐ_𝑏, and ℐ_𝑎,0∶=𝑠⁰_𝑎,𝑡⁰_𝑎,ℐ_𝑏,0∶=𝑠⁰_𝑏,𝑡⁰_𝑏.(55) By the corresponding intervals, respectively. It is evident that one of (𝐴_𝑎,2) and (𝐴_𝑏,2) implies (𝐴₂).
But (𝐴₂) cannot imply that there exists 𝑐₀∈ℐ such that both (𝐴_𝑎,2) and (𝐴_𝑏,2) hold.
(4) Several sufficient conditions for the definiteness condition can be given. The reader is referred to [23, Section 4].
For convenience, denote ℳ_𝜆∶=𝑦∈ℒ²_𝑊(ℐ)∶𝜏𝑦(𝑡)=𝜆𝑊(𝑡)𝑅𝑦(𝑡)for,𝑀𝑡∈ℐ_𝜆∶=𝑦^𝜋∈𝐿²_𝑊(ℐ)∶𝑦^𝜋,𝜆𝑦^𝜋∈𝐻(𝜏).(56)

Lemma 22. Assume that (𝐴₁) holds. For any 𝜆∈ℂ, dimℳ_𝜆=dim𝑀_𝜆 if and only if (𝐴₂) holds.

4. Characterizations of Minimal and Maximal Subspaces and Defect Indices of Minimal Subspaces

This section is divided into three subsections. In the first subsection, we give all the characterizations of the minimal subspaces generated by 1_𝜆 in ℐ, ℐ_𝑎, and ℐ_𝑏. In the second subsection, we study the defect indices of the minimal subspaces. In the third subsection, characterizations of the maximal subspaces are established.

4.1. Characterizations of the Minimal Subspaces

In this subsection, we study characterizations of the minimal subspaces generated by 1_𝜆 in ℐ, ℐ_𝑎, and ℐ_𝑏.

The following result is a direct consequence of Theorem 17.

Theorem 23. Assume that (𝐴₁) holds. Then 𝐻₀(𝜏), 𝐻_𝑏,0(𝜏), and 𝐻_𝑎,0(𝜏) are closed 𝐽-Hermitian subspace in (𝐿²_𝑊(ℐ))², (𝐿²_𝑊(ℐ_𝑏))², and (𝐿²_𝑊(ℐ_𝑎))², respectively.

Now, we introduce boundary forms on (𝐿²_𝑊(ℐ))², (𝐿²_𝑊(ℐ_𝑎))², and (𝐿²_𝑊(ℐ_𝑏))² by [∶]∶𝐿²_𝑊(ℐ)²×𝐿²_𝑊(ℐ)²𝑥⟶ℂ,_𝜋,𝑓^𝜋∶𝑦^𝜋,𝑔^𝜋=𝑓^𝜋,𝐽𝑦^𝜋−𝑥^𝜋,𝐽𝑔^𝜋,[∶]_𝑎∶𝐿²_𝑊ℐ_𝑎²×𝐿²_𝑊ℐ_𝑎²𝑥⟶ℂ,_𝜋,𝑓^𝜋∶𝑦^𝜋,𝑔^𝜋_𝑎=𝑓^𝜋,𝐽𝑦^𝜋_𝑎−𝑥^𝜋,𝐽𝑔^𝜋_𝑎,[∶]_𝑎∶𝐿²_𝑊ℐ_𝑏²×𝐿²_𝑊ℐ_𝑏²𝑥⟶ℂ,_𝜋,𝑓^𝜋∶𝑦^𝜋,𝑔^𝜋_𝑏=𝑓^𝜋,𝐽𝑦^𝜋_𝑏−𝑥^𝜋,𝐽𝑔^𝜋_𝑏.(57)

Lemma 24. Assume that (𝐴₁) holds. (1) If (𝐴₂) holds, then for any (𝑥,𝑓^𝜋),(𝑦,𝑔^𝜋)∈𝐻(𝜏), 𝑥,𝑓^𝜋∶𝑦,𝑔^𝜋=𝑥,𝑦(𝑏+1)−𝑥,𝑦(𝑎).(58)(2) If (𝐴_𝑎,2) holds, then for any (𝑥,𝑓^𝜋),(𝑦,𝑔^𝜋)∈𝐻_𝑎(𝜏), 𝑥,𝑓^𝜋∶𝑦,𝑔^𝜋_𝑎=𝑥,𝑦𝑐₀−𝑥,𝑦(𝑎).(59)(3) If (𝐴_𝑏,2) holds, then for any (𝑥,𝑓^𝜋),(𝑦,𝑔^𝜋)∈𝐻_𝑏(𝜏), 𝑥,𝑓^𝜋∶𝑦,𝑔^𝜋_𝑏=𝑥,𝑦(𝑏+1)−𝑥,𝑦𝑐₀.(60)

Proof. Since the proofs of (1)–(3) are similar, we only show that assertion (1) holds.
For any (𝑥,𝑓^𝜋),(𝑦,𝑔^𝜋)∈𝐻(𝜏), we have from (2) of Lemma 14 that 𝑓^𝜋,𝐽𝑦^𝜋−𝑥^𝜋,𝐽𝑔^𝜋=^𝑘_𝑡=𝑠𝑅𝐽₀𝑦^∗(𝑡)𝑊(𝑡)𝑅𝑓(𝑡)−𝑅𝐽₀𝑔^∗(𝑡)𝑊(𝑡)𝑅(𝑥𝑡=)()_{𝑘𝑡=𝑠}𝑅𝐽₀𝑦^∗(𝑡)𝜏(𝑥𝑡)−𝐽)(₀𝜏𝑦^∗(𝑡)𝑅(𝑥𝑡=𝑡)()𝑥,𝑦()_𝑠^𝑘+1(61) for any 𝑠<𝑘∈ℐ. This yields that lim_𝑡→𝑏(𝑥,𝑦)(𝑡) exists and is finite for any (𝑥,𝑓^𝜋),(𝑦,𝑔^𝜋)∈𝐻(𝜏). Similarly, it can be shown that lim_𝑡→𝑎(𝑥,𝑦)(𝑎) exists and is finite for any (𝑥,𝑓^𝜋),(𝑦,𝑔^𝜋)∈𝐻(𝜏). Hence, assertion (1) holds. The proof is complete.

Lemma 25. Assume that (𝐴₁) and (𝐴₂) hold. Then for any given finite subset ℐ₁=[𝑠,𝑘] with ℐ₀⊂ℐ₁⊂ℐ and for any given 𝛼,𝛽∈ℂ^2𝑛, there exists 𝑓={𝑓(𝑡)}^𝑘+1_𝑡=𝑠⊂ℂ^2𝑛 such that the following boundary value problem: 𝜏(𝑥)(𝑡)=𝑊(𝑡)𝑅𝑓(𝑡),𝑡∈ℐ₁,𝑥(𝑠)=𝛼,𝑥(𝑘+1)=𝛽(62) has a solution 𝑥={𝑥(𝑡)}^𝑘+1_𝑡=𝑠⊂ℂ^2𝑛.

Proof. Set 𝑥,𝑦^′∶=^𝑘_𝑡=𝑠𝑅𝑦^∗(𝑡)𝑊(𝑡)𝑅(𝑥)(𝑡),∀𝑥,𝑦∈𝑙([𝑠,𝑘]).(63) Let 𝜙_𝑗,1≤𝑗≤2𝑛, be the linearly independent solutions of system (1₀). Then we have rank𝜙_𝑖,𝜙_𝑗^′_{1≤𝑖,𝑗≤2𝑛}=2𝑛.(64) In fact, the linear algebraic system 𝜙_𝑖,𝜙_𝑗^′_{1≤𝑖,𝑗≤2𝑛}𝐶=0,(65) where 𝐶=(𝑐₁,𝑐₂,…,𝑐_2𝑛)^𝑇∈ℂ^2𝑛, can be written as 𝜙_𝑖,^2𝑛_𝑗=1𝑐_𝑗𝜙_𝑗^′=0,1≤𝑖≤2𝑛,(66) which yields ^2𝑛_𝑗=1𝑐_𝑗𝜙_𝑗,^2𝑛_𝑗=1𝑐_𝑗𝜙_𝑗^′=0.(67) Since ∑^2𝑛_𝑗=1𝑐_𝑗𝜙_𝑗 is a solution of system (1₀), it follows from (𝐴₂) that ∑^2𝑛_𝑗=1𝑐_𝑗𝜙_𝑗=0. Then 𝐶=0; that is, (65) has only a zero solution. Consequently, (64) holds.
Let 𝛼, 𝛽 be any given vectors in ℂ^2𝑛. By (64), the linear algebraic system 𝜙_𝑖,𝜙_𝑗^′𝑇_{1≤𝑖,𝑗≤2𝑛}𝐶=𝜙₁(𝑘+1),𝜙₂(𝑘+1),…,𝜙_2𝑛(𝑘+1)^∗𝒥𝛽(68) has a unique solution 𝐶₁∈ℂ^2𝑛. Set 𝜓₁=(𝜙₁,𝜙₂,…,𝜙_2𝑛)𝐶₁. It follows from (68) that 𝜓₁,𝜙_𝑖^′=𝜙^∗_𝑖(𝑘+1)𝒥𝛽,1≤𝑖≤2𝑛.(69) Let 𝑢(𝑡) be a solution of the following initial value problem: 𝜏(𝑥)(𝑡)=𝑊(𝑡)𝑅𝜓₁(𝑡)[],𝑥(𝑠),𝑡∈𝑠,𝑘=0.(70) Since 𝜏(𝜙_𝑖)(𝑡)=0 for 𝑡∈ℐ and 1≤𝑖≤2𝑛, we get by (70) and (2) of Lemma 14 that 𝜓₁,𝜙_𝑖^′=^𝑘_𝑡=𝑠𝑅𝜙_𝑖^∗(𝑡)𝜏(𝑢)(𝑡)−𝜏𝜙_𝑖^∗(𝑡)𝑅(𝑢𝑡=)()𝑢,𝐽₀𝜙_𝑖(𝑡)_𝑠^𝑘+1=𝜙^∗_𝑖(𝑘+1)𝒥𝑢(𝑘+1),1≤𝑖≤2𝑛.(71) Since 𝜙₁,𝜙₂,…,𝜙_2𝑛 are linearly independent in 𝑙(ℐ), we get from (69) and (71) that 𝑢(𝑘+1)=𝛽. So, 𝑢(𝑡) is a solution of the following boundary value problem: 𝜏(𝑥)(𝑡)=𝑊(𝑡)𝑅𝜓₁(𝑡)[],𝑥(𝑠)(),𝑡∈𝑠,𝑘=0,𝑥𝑘+1=𝛽.(72)
On the other hand, the linear algebraic system 𝜙_𝑖,𝜙_𝑗^′𝑇_{1≤𝑖,𝑗≤2𝑛}𝐶=𝜙₁(𝑠),𝜙₂(𝑠),…,𝜙_2𝑛(𝑠)^∗𝐽𝛼(73) has a unique solution 𝐶₂∈ℂ^2𝑛 by (64). Set 𝜓₂=(𝜙₁,𝜙₂,…,𝜙_2𝑛)𝐶₂. Then, by (73) 𝜓₂,𝜙_𝑖^′=𝜙^∗_𝑖(𝑠)𝐽𝛼,1≤𝑖≤2𝑛.(74) Let 𝑣(𝑡) be a solution of the following initial value problem: 𝜏(𝑥)(𝑡)=−𝑊(𝑡)𝑅𝜓₂(𝑡),𝑥(𝑘+1)=0,𝑡∈[𝑠,𝑘].(75) Since 𝜏(𝜙_𝑖)(𝑡)=0 for 𝑡∈ℐ and 1≤𝑖≤2𝑛, we get by (2) of Lemma 14 and (75) that 𝜓₂,𝜙_𝑖^′=𝜙^∗_𝑖(𝑠)𝐽𝑣(𝑠),1≤𝑖≤2𝑛,(76) which, together with (74), implies that 𝑣(𝑠)=𝛼. So, 𝑣(𝑡) is a solution of the following boundary value problem: 𝜏(𝑥)(𝑡)=−𝑊(𝑡)𝑅𝜓₂(𝑡)[],𝑥(𝑠)(),𝑡∈𝑠,𝑘=𝛼,𝑥𝑘+1=0.(77) Set 𝜓=𝜓₁−𝜓₂ and 𝜙=𝑢+𝑣. Then 𝜙 is a solution of the boundary value problem (62). The proof is complete.

Remark 26. Lemma 25 is called a patch lemma. Based on Lemma 25, any two elements of 𝐻(𝜏) (𝐻_𝑏(𝜏), 𝐻_𝑎(𝜏), resp.) can be patched up to construct another new element of 𝐻(𝜏) (𝐻_𝑏(𝜏), 𝐻_𝑎(𝜏), resp.). In particular, (1) if (𝐴_𝑏,2) holds, we can take ℐ₁=[𝑐₀,𝑡⁰_𝑏], 𝛼=𝑒_𝑖∶=(0,…,0_𝑖−1,1,0,…,0)^𝑇, and 𝛽=0, 1≤𝑖≤2𝑛. Then there exist (𝑧^𝑖_𝑏,(ℎ^𝜋)^𝑖_𝑏)∈𝐻_𝑏(𝜏) satisfying 𝑧^𝑖_𝑏𝑐₀=𝑒_𝑖;𝑧^𝑖_𝑏(𝑡)=0,𝑡≥𝑡⁰_𝑏+1,1≤𝑖≤2𝑛;(78)(2) if (𝐴_𝑎,2) holds, we take ℐ₁=[𝑠⁰_𝑎,𝑐₀−1], 𝛼=0, and 𝛽=𝑒_𝑖. Then there exist (𝑧^𝑖_𝑎,(ℎ^𝜋)^𝑖_𝑎)∈𝐻_𝑎(𝜏) satisfying 𝑧^𝑖_𝑎𝑐₀=𝑒_𝑖;𝑧^𝑖_𝑎(𝑡)=0,𝑡≤𝑠⁰_𝑎,1≤𝑖≤2𝑛;(79)(3) if both (𝐴_𝑏,2) and (𝐴_𝑎,2) hold, then there exist (𝑧_𝑖,ℎ^𝜋_𝑖)∈𝐻(𝜏) satisfying 𝑧_𝑖𝑐₀=𝑒_𝑖;𝑧_𝑖(𝑡)=0for𝑡≤𝑠⁰_𝑎,𝑡≥𝑡⁰_𝑏+1,1≤𝑖≤2𝑛.(80)The above auxiliary elements (𝑧^𝑖_𝑏,(ℎ^𝜋)^𝑖_𝑏), (𝑧^𝑖_𝑎,(ℎ^𝜋)^𝑖_𝑎), and (𝑧_𝑖,ℎ^𝜋_𝑖)(1≤𝑖≤2𝑛) will be very useful in the sequent discussions.

Theorem 27. Assume that (𝐴₁) holds.(1) If (𝐴₂) holds, then 𝐻₀(𝜏)=𝑥,𝑓^𝜋(𝜏)∶()=(𝑎)(𝜏.∈𝐻𝑥,𝑦𝑏+1𝑥,𝑦=0∀𝑦∈Dom𝐻)(81) In particular, if ℐ=[𝑎,𝑏], then 𝐻₀₀(𝜏)=𝑥,𝑓^𝜋∈𝐻(𝜏)∶𝑥(𝑎)=𝑥(𝑏+1)=0.(82)(2) If (𝐴_𝑏,2) holds, then 𝐻_𝑏,0(𝜏)=𝑥,𝑓^𝜋∈𝐻_𝑏(𝜏)∶𝑥𝑐₀=0,𝑥,𝑦(𝑏+1)=0∀𝑦∈Dom𝐻_𝑏(𝜏).(83)(3) If (𝐴_𝑎,2) holds, then 𝐻_𝑎,0(𝜏)=𝑥,𝑓^𝜋∈𝐻_𝑎(𝜏)∶𝑥𝑐₀=0,𝑥,𝑦(𝑎)=0∀𝑦∈Dom𝐻_𝑎(𝜏).(84)

Proof. We first show that assertion (1) holds. By Lemmas 8 and 24, and Theorem 17, one has 𝐻₀(𝜏)=𝑥,𝑓^𝜋(𝜏)∶()=(𝑎)(𝜏.∈𝐻𝑥,𝑦𝑏+1𝑥,𝑦∀𝑦∈Dom𝐻)(85) For convenience, denote 𝐻⁰(𝜏)=𝑥,𝑓^𝜋(𝜏)∶()=(𝑎)(𝜏.∈𝐻𝑥,𝑦𝑏+1𝑥,𝑦=0∀𝑦∈Dom𝐻)(86) Clearly, 𝐻⁰(𝜏)⊂𝐻₀(𝜏). We now show that 𝐻₀(𝜏)⊂𝐻⁰(𝜏). Fix any (𝑥,𝑓^𝜋)∈𝐻₀(𝜏). It follows from (85) that for all (𝑦,𝑔^𝜋)∈𝐻(𝜏), 𝑥,𝑦(𝑏+1)=𝑥,𝑦(𝑎).(87) For any given (𝑦,𝑔^𝜋)∈𝐻(𝜏), by Remark 26 there exists (𝑧,ℎ^𝜋)∈𝐻(𝜏) such that 𝑧(𝑡)=0,𝑡≤𝑠₀;𝑧(𝑡)=𝑦(𝑡),𝑡≥𝑡₀+1.(88) Thus, it follows from (87) that (𝑥,𝑦)(𝑏+1)=(𝑥,𝑧)(𝑏+1)=(𝑥,𝑧)(𝑎)=0, and consequently (𝑥,𝑦)(𝑎)=0 for all (𝑦,𝑔^𝜋)∈𝐻(𝜏).
In the case that ℐ=[𝑎,𝑏], it is clear that 𝐻₀₀(𝜏)=𝑥,𝑓^𝜋∈𝐻(𝜏)∶𝑥(𝑎)=𝑥(𝑏+1)=0.(89) So it remains to show that 𝐻₀(𝜏)=𝐻₀₀(𝜏). It suffices to show that 𝑥(𝑎)=𝑥(𝑏+1)=0 for any (𝑥,𝑓^𝜋)∈𝐻₀(𝜏). Fix any (𝑥,𝑓^𝜋)∈𝐻₀(𝜏), and let ℐ₀=ℐ, 𝛼=𝑒_𝑖, 1≤𝑖≤2𝑛, and 𝛽=0. Then by Lemma 25, there exist (𝑦_𝑖,𝑔^𝜋_𝑖)∈𝐻(𝜏) with 𝑦_𝑖(𝑎)=𝑒_𝑖 and 𝑦_𝑖(𝑏+1)=0. Inserting these 𝑦_𝑖 into (87) one has that 𝑥(𝑎)=0. Similarly, one can show that 𝑥(𝑏+1)=0. Thus 𝐻₀(𝜏)=𝐻₀₀(𝜏). Therefore, assertion (1) has been shown.
With similar arguments, one can show that assertion (2) and (3) hold by using (78) and (79), separately. This completes the proof.

4.2. Defect Indices of Minimal Subspaces

In this subsection, we first give a valued range of the defect indices of 𝐻_𝑏,0(𝜏) and 𝐻_𝑎,0(𝜏) and then discuss the relationship among the defect indices of 𝐻_𝑏,0(𝜏), 𝐻_𝑎,0(𝜏), and 𝐻₀(𝜏).

For briefness, denote 𝑑∶=𝑑𝐻₀(𝜏),𝑑_𝑏∶=𝑑𝐻_𝑏,0(𝜏),𝑑_𝑎∶=𝑑𝐻_𝑎,0(𝜏).(90) For any 𝜆∈ℂ, let ℳ_𝑏,𝜆, 𝑀_𝑏,𝜆, ℳ_𝑎,𝜆, and 𝑀_𝑎,𝜆 be defined as (56) with ℐ replaced by ℐ_𝑏 and ℐ_𝑎, respectively.

The following results are obtained.

Theorem 28. Assume that (𝐴₁) holds.(1) If (𝐴_𝑏,2) holds and Γ(𝐻_𝑏,0(𝜏))≠∅, then 𝑑_𝑏=dimℳ_𝑏,𝜆 for any 𝜆∈Γ(𝐻_𝑏,0(𝜏)), and 𝑛≤𝑑_𝑏≤2𝑛. (2) If (𝐴_𝑎,2) holds and Γ(𝐻_𝑎,0(𝜏))≠∅, then 𝑑_𝑎=dimℳ_𝑎,𝜆 for any 𝜆∈Γ(𝐻_𝑎,0(𝜏)), and 𝑛≤𝑑_𝑎≤2𝑛.

Proof. Since the method of the proofs is the same, we only give the proof of assertion (1).
For any 𝜆∈Γ(𝐻_𝑏,0(𝜏)), it follows from Lemma 11 and Theorem 18 that 𝑑_𝑏=dimKer𝐻_𝑏(𝛿)−𝜆.(91) On the other hand, by using Lemma 5 and Theorems 17 and 18, one has that dimKer𝐻_𝑏(𝜏)−𝜆=dimKer𝐻_𝑏(𝛿)−𝜆.(92) It is clear that 𝑀_𝑏,𝜆=Ker𝐻_𝑏(𝜏)−𝜆.(93) Combining (91)–(93), one has 𝑑_𝑏=dim𝑀_𝑏,𝜆. This, together with Lemma 22, implies that 𝑑_𝑏=dimℳ_𝑏,𝜆. In addition, it has been shown in [21] that 𝑛≤dimℳ_𝑏,𝜆≤2𝑛 for any 𝜆∈ℂ. So assertion (1) is true. The proof is complete.

Next, we discuss relationship among defect indices of 𝐻₀(𝜏), 𝐻_𝑎,0(𝜏), and 𝐻_𝑏,0(𝜏). For convenience, denote 𝑦_𝑎∶=𝑦(𝑡)^𝑐0_𝑡=𝑎,𝑦_𝑏∶=𝑦(𝑡)^𝑏+1_𝑡=𝑐0(94) for any 𝑦∈ℒ²_𝑊(ℐ). It is evident that 𝑦_𝑎∈ℒ²_𝑊(ℐ_𝑎), 𝑦_𝑏∈ℒ²_𝑊(ℐ_𝑏), and 𝑦²=𝑦_𝑎²_𝑎+𝑦_𝑏²_𝑏.(95) On the other hand, for any 𝑦_𝑎=(𝑢^𝑇_𝑎,𝑣^𝑇_𝑎)^𝑇∈ℒ²_𝑊(ℐ_𝑎) and 𝑦_𝑏=(𝑢^𝑇_𝑏,𝑣^𝑇_𝑏)^𝑇∈ℒ²_𝑊(ℐ_𝑏), we define 𝑦 by 𝑦(𝑡)=𝑦_𝑎(𝑡),𝑡≤𝑐₀𝑢−1,_𝑇𝑎𝑐₀,𝑣^𝑇_𝑏𝑐₀^𝑇,𝑡=𝑐₀,𝑦_𝑏(𝑡),𝑡≥𝑐₀+1.(96) Then (95) still holds, and consequently 𝑦∈ℒ²_𝑊(ℐ). Furthermore, it is clear that 𝑅𝑦(𝑡)=𝑅𝑦_𝑎(𝑡),𝑡≤𝑐₀𝑅𝑦(𝑡)𝑦−1;=𝑅_𝑏(𝑡),𝑡≥𝑐₀.(97)

The following result can be easily verified. So we omit its proof.

Lemma 29. Assume that (𝐴₁), (𝐴_𝑎,2), and (𝐴_𝑏,2) hold. (1) If (𝑦,𝑔^𝜋)∈𝐻(𝜏), then (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏(𝜏) and (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎(𝜏). (2) If (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎(𝜏) and (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏(𝜏) with 𝑦_𝑎(𝑐₀)=𝑦_𝑏(𝑐₀), then (𝑦,𝑔^𝜋)∈𝐻(𝜏).

Let 𝐻₀(𝜏) be the restriction of subspace 𝐻₀(𝜏), defined by 𝐻₀(𝜏)=𝑦,𝑔^𝜋∈𝐻₀(𝜏)∶𝑦𝑐₀=0.(98)

Lemma 30. Assume that (𝐴₁), (𝐴_𝑎,2), and (𝐴_𝑏,2) hold. Then (1)(𝑦,𝑔^𝜋)∈𝐻₀(𝜏) if and only if (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎,0(𝜏) and (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏,0(𝜏); (2)(𝑦,𝑔^𝜋)∈(𝐻₀(𝜏))^∗_𝐽 if and only if (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎(𝜏) and (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏(𝜏).

Proof. (1) We first consider the necessity. Fix any (𝑦,𝑔^𝜋)∈𝐻₀(𝜏). Let 𝑦_𝑎, 𝑦_𝑏, 𝑔_𝑎, and 𝑔_𝑏 be defined as (94). Then 𝑦_𝑎(𝑐₀)=𝑦_𝑏(𝑐₀)=0. By (1) of Lemma 29 one has that (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎(𝜏),(𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏(𝜏). In addition, for any (𝑥_𝑎,𝑓^𝜋_𝑎)∈𝐻_𝑎(𝜏), it follows from Remark 26 that there exits (𝑧,ℎ^𝜋)∈𝐻(𝜏) with 𝑧(𝑡)=𝑥_𝑎(𝑡),𝑡≤𝑐₀,𝑧(𝑡)=0,𝑡≥𝑡⁰_𝑏+1.(99) So one has 𝑦_𝑎,𝑥_𝑎(𝑎)=𝑦,𝑧(𝑎)=0,(100) which implies that (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎,0(𝜏) by the arbitrariness of (𝑥_𝑎,𝑓^𝜋_𝑎)∈𝐻_𝑎(𝜏) and (3) of Theorem 27. With a similar argument, one can show (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏,0(𝜏).
Next, we consider the sufficiency. Fix any (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎,0(𝜏) and (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏,0(𝜏). Let 𝑦 and 𝑔 be defined by (96). By (2) and (3) of Theorem 27 one has that 𝑦_𝑎(𝑐₀)=𝑦_𝑏(𝑐₀)=0. So 𝑦(𝑐₀)=0. It follows from (2) of Lemma 29 that (𝑦,𝑔^𝜋)∈𝐻(𝜏). For any (𝑥,𝑓^𝜋)∈𝐻(𝜏), it follows from (1) of Lemma 29 that (𝑥_𝑎,𝑓^𝜋_𝑎)∈𝐻_𝑎(𝜏) and (𝑥_𝑏,𝑓^𝜋_𝑏)∈𝐻_𝑏(𝜏). Thus one has by (2) and (3) of Theorem 27 that 𝑦,𝑥(𝑎)=𝑦_𝑎,𝑥_𝑎(𝑎)()=𝑦=0,𝑦,𝑥𝑏+1_𝑏,𝑥_𝑏(𝑏+1)=0.(101) This implies that (𝑦,𝑔^𝜋)∈𝐻₀(𝜏) by (1) of Theorem 27, and consequently, (𝑦,𝑔^𝜋)∈𝐻₀(𝜏).
(2) We first consider the necessity. Fix any (𝑦,𝑔^𝜋)∈(𝐻₀(𝜏))^∗_𝐽. Let 𝑦_𝑎, 𝑦_𝑏, 𝑔_𝑎, and 𝑔_𝑏 be defined as (94). Then (𝑦_𝑎,𝑔^𝜋_𝑎)∈(𝐿²_𝑊(ℐ_𝑎))² and (𝑦_𝑏,𝑔^𝜋_𝑏)∈(𝐿²_𝑊(ℐ_𝑏))².
Set 𝑥_𝑏(𝑡)=𝑓_𝑏(𝑡)=0 for 𝑡∈ℐ^∗_𝑏, where ℐ^∗_𝑏 is defined by (19) with ℐ replaced by ℐ_𝑏. It is clear that (𝑥_𝑏,𝑓^𝜋_𝑏)∈𝐻_𝑏,0(𝜏). For any (𝑥_𝑎,𝑓^𝜋_𝑎)∈𝐻_𝑎,0(𝜏), let 𝑥 and 𝑓 be defined by (96). Then by the above result (1) one has that (𝑥,𝑓^𝜋)∈𝐻₀(𝜏). It follows that 0=⟨𝑓,𝐽𝑦⟩−𝑥,𝐽𝑔=𝑔_𝑎,𝐽𝑦_𝑎𝑎−𝑥_𝑎,𝐽𝑔_𝑎𝑎.(102) By the arbitrariness of (𝑥_𝑎,𝑓^𝜋_𝑎)∈𝐻_𝑎,0(𝜏), one has that (𝑦_𝑎,𝑔^𝜋_𝑎)∈(𝐻_𝑎,0(𝜏))^∗_𝐽=𝐻_𝑎(𝜏) by Theorem 17. With a similar argument one can show (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏(𝜏).
Next, we consider the sufficiency. Fix any (𝑦_𝑎,𝑔^𝜋_𝑎)∈𝐻_𝑎(𝜏) and (𝑦_𝑏,𝑔^𝜋_𝑏)∈𝐻_𝑏(𝜏). Let 𝑦 and 𝑔 be defined by (96). For any (𝑥,𝑓^𝜋)∈𝐻₀(𝜏), it follows from the above result (1) that (𝑥_𝑎,𝑓^𝜋_𝑎)∈𝐻_𝑎,0(𝜏) and (𝑥_𝑏,𝑓^𝜋_𝑏)∈𝐻_𝑏,0(𝜏). So one has by Theorem 17 that 𝑓,𝐽₀𝑦−𝑥,𝐽₀𝑔=𝑔_𝑎,𝐽₀𝑦_𝑎𝑎−𝑥_𝑎,𝐽₀𝑔_𝑎𝑎+𝑔_𝑏,𝐽₀𝑦_𝑏𝑏−𝑥_𝑏,𝐽₀𝑔_𝑏𝑏=0.(103) This yields that (𝑦,𝑔^𝜋)∈(𝐻₀(𝜏))^∗_𝐽. The whole proof is complete.

Lemma 31. Assume that (𝐴₁), (𝐴_𝑎,2), and (𝐴_𝑏,2) hold. Then Γ(𝐻₀(𝜏))⊂Γ(𝐻₀(𝜏)) and Γ(𝐻_𝑏,0(𝜏))∩Γ(𝐻_𝑎,0(𝜏))=Γ(𝐻₀(𝜏)).

Proof. The first assertion holds because 𝐻₀(𝜏)⊂𝐻₀(𝜏), and the second assertion can be proved by (1) of Lemma 30 and (95). The proof is complete.

In the following of the present paper, we assume that (𝐴₃)Γ(𝐻₀(𝜏))≠∅.

By Definition 10, one has that if Γ(𝐻₀(𝜏))=∅, then 𝜌(𝐻₀(𝜏))=∅, and consequently 𝜎(𝐻₀(𝜏))=ℂ. We do not consider this case in the present paper.

Theorem 32. Assume that (𝐴₁), (𝐴_𝑎,2), (𝐴_𝑏,2), and (𝐴₃) hold. Then 𝑑=𝑑_𝑎+𝑑_𝑏−2𝑛.(104)

Proof. The proof is divided into two steps.
Step  1. We show that 𝑑𝐻₀(𝜏)=𝑑_𝑎+𝑑_𝑏.(105)
It follows from Γ(𝐻₀(𝜏))≠∅ and Lemma 31 that Γ(𝐻_𝑏,0(𝜏))∩Γ(𝐻_𝑎,0(𝜏))≠∅. This, together with (𝐴_𝑎,2), (𝐴_𝑏,2), and Theorem 28, implies that for any 𝜆∈Γ(𝐻_𝑏,0(𝜏))∩Γ(𝐻_𝑎,0(𝜏)), 1_𝜆 has just 𝑑_𝑏 linearly independent solutions 𝑦^𝑗_𝑏∈ℒ²_𝑊(ℐ_𝑏), 1≤𝑗≤𝑑_𝑏, and 1_𝜆 has just 𝑑_𝑎 linearly independent solutions 𝑥^𝑗_𝑎∈ℒ²_𝑊(ℐ_𝑎), 1≤𝑗≤𝑑_𝑎; that is, (𝑦^𝑗_𝑏,𝜆(𝑦^𝑗_𝑏)^𝜋)∈𝐻_𝑏(𝜏) for 1≤𝑗≤𝑑_𝑏 and (𝑥^𝑗_𝑎,𝜆(𝑥^𝑗_𝑎)^𝜋)∈𝐻_𝑎(𝜏) for 1≤𝑗≤𝑑_𝑎.
Set 𝑥^𝑗_𝑏(𝑡)=0 for 𝑡∈ℐ^∗_𝑏. It is clear that (𝑥^𝑗_𝑏,𝜆(𝑥^𝑗_𝑏)^𝜋)∈𝐻_𝑏(𝜏) for 1≤𝑗≤𝑑_𝑎. Let 𝑥_𝑗, 1≤𝑗≤𝑑_𝑎, be defined by (96). Then it follows from (2) of Lemma 30 that (𝑥_𝑗,𝜆𝑥^𝜋_𝑗)∈(𝐻₀(𝜏))^∗_𝐽. Similarly, set 𝑦^𝑗_𝑎(𝑡)=0 for 𝑡∈ℐ^∗_𝑎. Then one has (𝑦_𝑗,𝜆𝑦^𝜋_𝑗)∈(𝐻₀(𝜏))^∗_𝐽 for 1≤𝑗≤𝑑_𝑏. It is evident that 𝑥₁,…,𝑥_𝑑𝑎,𝑦₁,…,𝑦_𝑑𝑏∈Dom(𝐻₀(𝜏))^∗_𝐽 are linearly independent.
On the other hand, for any (𝑦,𝜆𝑦^𝜋)∈(𝐻₀(𝜏))^∗_𝐽, it follows from (2) of Lemma 31 that (𝑦_𝑎,𝜆𝑦^𝜋_𝑎)∈𝐻_𝑎(𝜏) and (𝑦_𝑏,𝜆𝑦^𝜋_𝑏)∈𝐻_𝑏(𝜏); that is, 𝑦_𝑎∈ℒ²_𝑊(ℐ_𝑎) is a solution of 1_𝜆 in ℐ_𝑎, and 𝑦_𝑏∈ℒ²_𝑊(ℐ_𝑏) is a solution of 1_𝜆 in ℐ_𝑏. Therefore, there exist unique 𝑐_𝑗∈ℂ(1≤𝑗≤𝑑_𝑎) and 𝑑_𝑗∈ℂ(1≤𝑗≤𝑑_𝑏) such that 𝑦_𝑎=^𝑑𝑎_𝑗=1𝑐_𝑗𝑥^𝑗_𝑎,𝑦_𝑏=^𝑑𝑏_𝑗=1𝑑_𝑗𝑦^𝑗_𝑏.(106) Noting the constructions of 𝑥_𝑗 and 𝑦_𝑗 by (96), one has that 𝑦=^𝑑𝑎_𝑗=1𝑐_𝑗𝑥_𝑗+^𝑑𝑏_𝑗=1𝑑_𝑗𝑦_𝑗.(107) This, together with Lemma 11, implies that (105) holds.
Step  2. We show that 𝑑=𝑑𝐻₀(𝜏)−2𝑛.(108)
It is evident that (𝑧_𝑖,ℎ^𝜋_𝑖)∈𝐻₀(𝜏), 1≤𝑖≤2𝑛, where (𝑧_𝑖,ℎ^𝜋_𝑖) are defined by (80). We claim that 𝐻₀(𝜏)=𝐻₀(𝜏)∔span𝑧₁,ℎ^𝜋₁,𝑧₂,ℎ^𝜋₂,…,𝑧_2𝑛,ℎ^𝜋_2𝑛.(109) In fact, for each (𝑦,𝑔^𝜋)∈𝐻₀(𝜏), set 𝐶₀=(𝑐₁,𝑐₂,…,𝑐_2𝑛)^𝑇=𝑦(𝑐₀). Let 𝑦₀=𝑦−^2𝑛_𝑖=1𝑐_𝑖𝑧_𝑖,𝑔₀=𝑔−^2𝑛_𝑖=1𝑐_𝑖ℎ_𝑖.(110) Then 𝑦,𝑔^𝜋=𝑦₀,𝑔^𝜋₀+^2𝑛_𝑖=1𝑐_𝑖𝑧_𝑖,ℎ^𝜋_𝑖,𝑦₀,𝑔^𝜋₀∈𝐻₀(𝜏).(111) It can be easily verified that this decomposition is unique. Hence, (109) holds.
For any 𝜆∈Γ(𝐻₀(𝜏)), we claim that Ran𝐻₀(𝜏)−𝜆𝐼=Ran𝐻₀(𝜏)ℎ−𝜆𝐼∔span^𝜋₁−𝜆𝑧^𝜋₁,ℎ^𝜋₂−𝜆𝑧^𝜋₂ℎ,…,_𝜋2𝑛−𝜆𝑧^𝜋_2𝑛.(112) Since (109) holds, it suffices to show that Ran𝐻₀(𝜏)−𝜆𝐼∩spanℎ^𝜋₁−𝜆𝑧^𝜋₁,ℎ^𝜋₂−𝜆𝑧^𝜋₂ℎ,…,_𝜋2𝑛−𝜆𝑧^𝜋_2𝑛={0}.(113) Suppose that (𝑦,𝑔^𝜋)∈𝐻₀(𝜏) and 𝑐₁,𝑐₂,…,𝑐_2𝑛∈ℂ satisfy 𝑔^𝜋−𝜆𝑦^𝜋+^2𝑛_𝑗=1𝑐_𝑗ℎ^𝜋_𝑗−𝜆𝑧^𝜋_𝑗=0,(114) that is, 𝑔^𝜋+^2𝑛_𝑗=1𝑐_𝑗ℎ^𝜋_𝑗=𝜆𝑦^𝜋+^2𝑛_𝑗=1𝑐_𝑗𝑧^𝜋_𝑗.(115) Since (𝑦,𝑔^𝜋)∈𝐻₀(𝜏) and (𝑧_𝑖,ℎ^𝜋_𝑖)∈𝐻₀(𝜏), it follows that (𝑦+∑^2𝑛_𝑗=1𝑐_𝑗𝑧_𝑗,𝜆(𝑦^𝜋+∑^2𝑛_𝑗=1𝑐_𝑗𝑧^𝜋_𝑗))∈𝐻₀(𝜏). Since 𝜆∈Γ(𝐻₀(𝜏)), it yields 𝑦+^2𝑛_𝑗=1𝑐_𝑗𝑧_𝑗=0,𝑔^𝜋+^2𝑛_𝑗=1𝑐_𝑗ℎ^𝜋_𝑗=0,(116) which, together with (109), implies that 𝑐_𝑗=0 for 1≤𝑗≤2𝑛, and consequently 𝑔^𝜋=𝑦^𝜋=0. This yields that (113) holds.
Since 𝐻₀(𝜏) and 𝐻₀(𝜏) are closed 𝐽-Hermitian subspaces, it follows that Ran(𝐻₀(𝜏)−𝜆) and Ran(𝐻₀(𝜏)−𝜆) are closed subspaces in 𝐿²_𝑊(ℐ), respectively. Hence, there exists a closed subspace 𝑄 in 𝐿²_𝑊(ℐ) such that Ran𝐻₀(𝜏)−𝜆𝐼=Ran𝐻₀(𝜏)−𝜆𝐼⊕𝑄.(117) In addition, again by the fact that 𝜆∈Γ(𝐻_𝑏,0(𝜏))∩Γ(𝐻_𝑎,0(𝜏)), it follows that {ℎ^𝜋_𝑗−𝜆𝑧^𝜋_𝑗}^2𝑛_𝑗=1 are linearly independent in 𝐿²_𝑊(ℐ). It follows from (113) that dim𝑄=2𝑛. Consequently, dimRan𝐻₀(𝜏)−𝜆𝐼^⟂=dimRan𝐻₀(𝜏)−𝜆𝐼^⟂−2𝑛.(118) This yields that (108) holds. It follows from (105) and (108) that (104) holds. The proof is complete.

Remark 33. Theorem 32 (formula (104)) generalizes the classical result for 2𝑛th order ordinary differential equations that go back to the classical work by Akhiezer and Glazman [1, Theorem 3 in Appendix 2]. To the case of symmetric Hamiltonian systems, formula (104) was extended in [15].
So it follows from Theorems 28 and 32 that 𝑑=0 if and only if 𝑑_𝑎=𝑑_𝑏=𝑛, and 𝑑=2𝑛 if and only if 𝑑_𝑎=𝑑_𝑏=2𝑛. The following definition is obtained.

Definition 34. Assume that (𝐴₁), (𝐴₂), and (𝐴_𝑏,2) hold. Then 1_𝜆 is said to be in the limit 𝑑_𝑏 case at 𝑡=𝑏. In the special case that 𝑑_𝑏=𝑛, 1_𝜆 is said to be in the limit point case (𝑙.𝑝.𝑐.) at 𝑡=𝑏, and in the other special case that 𝑑_𝑏=2𝑛, 1_𝜆 is said to be in the limit circle case (𝑙.𝑐.𝑐.) at 𝑡=𝑏.
The same definition can be given at 𝑡=𝑎 provided that (𝐴_𝑎,2) holds.

4.3. Characterizations of 𝐻_𝑎(𝜏) and 𝐻_𝑏(𝜏)

In this subsection, we characterize the maximal subspaces 𝐻_𝑎(𝜏) and 𝐻_𝑏(𝜏). We first consider characterization of 𝐻_𝑏(𝜏). Assume that (𝐴_𝑏,2) holds and Γ(𝐻_𝑏,0(𝜏))≠∅. Let 𝜆∈Γ(𝐻_𝑏,0(𝜏)). It follows from the proof of Lemma 11 that 𝐻_𝑏(𝜏)=𝐻_𝑏,0(𝜏)∔𝑈₁∔𝑈₂,(119) where 𝑈₁={(𝑥^𝑗_𝑏,𝜆(𝑥^𝑗_𝑏)^𝜋)∈𝐻_𝑏(𝜏)∶1≤𝑗≤𝑑_𝑏} and 𝑈₂={(𝑦^𝑗_𝑏,(𝑔^𝑗_𝑏)^𝜋)∈𝐻_𝑏(𝜏)∶1≤𝑗≤𝑑_𝑏}, and {(𝑦^𝑗_𝑏,(𝑔^𝑗_𝑏)^𝜋)}^𝑑𝑏_𝑗=1 are linearly independent (mod 𝑈₁). For convenience, denote 𝜒^𝑖_𝑏(𝑡)∶=𝑥^𝑖_𝑏(𝑡),𝜒^𝑑𝑏_𝑏^+𝑗(𝑡)∶=𝑦^𝑗_𝑏(𝑡),1≤𝑖≤𝑑_𝑏,1≤𝑗≤𝑑_𝑏.(120) Clearly, 𝜒^𝑖_𝑏∈Dom𝐻_𝑏(𝜏), 1≤𝑖≤2𝑑_𝑏. So (𝜒^𝑖_𝑏,𝜒^𝑗_𝑏)(𝑏+1) is finite for all 1≤𝑖, 𝑗≤2𝑑_𝑏. Then the following result can be directly derived from (119) and (120).

Lemma 35. Assume that (𝐴₁) and (𝐴_𝑏,2) hold, and Γ(𝐻_𝑏,0(𝜏))≠∅. Then every 𝑦∈Dom𝐻_𝑏(𝜏) can be expressed as 𝑦=𝑦⁰_𝑏+^2𝑑𝑏_𝑖=1𝑎_𝑖𝜒^𝑖_𝑏,(121) where 𝑦⁰_𝑏∈Dom𝐻_𝑏,0(𝜏) and 𝑎_𝑖∈ℂ.

By Lemma 35, 𝑧^𝑖_𝑏, defined by (78), can be uniquely expressed as 𝑧^𝑖_𝑏=𝑦_𝑏^𝑖,0+^2𝑑𝑏_𝑗=1𝑎_𝑖𝑗𝜒^𝑗_𝑏,1≤𝑖≤2𝑛,(122) where 𝑦_𝑏^𝑖,0∈Dom𝐻_𝑏,0(𝜏) and 𝑎_𝑖𝑗∈ℂ. Denote 𝐸∶=𝑎_{𝑖𝑗2𝑛×2𝑑𝑏},𝐹₁∶=𝜒^𝑖_𝑏,𝜒^𝑗_𝑏(𝑏+1)_{1≤𝑖,𝑗≤2𝑑𝑏}.(123)

Lemma 36. Assume that (𝐴₁) and (𝐴_𝑏,2) hold, and Γ(𝐻_𝑏,0(𝜏))≠∅. Then rank𝐸=2𝑛 and rank𝐹₁=2𝑑_𝑏−2𝑛. Furthermore, we can rearrange the order of 𝜒¹_𝑏,𝜒²_𝑏,…,𝜒^𝑑𝑏_𝑏 such that rank𝜒^𝑖_𝑏,𝜒^𝑗_𝑏(𝑏+1)_{1≤𝑖≤2𝑑𝑏−2𝑛,1≤𝑗≤𝑑𝑏}=2𝑑_𝑏−2𝑛.(124)

Proof. It follows from (122) that 𝑧¹_𝑏,𝑧²_𝑏,…,𝑧_𝑏^2𝑛=𝑦_𝑏^1,0,𝑦_𝑏^2,0,…,𝑦_𝑏^2𝑛,0+𝜒¹_𝑏,𝜒²_𝑏,…,𝜒^2𝑑𝑏_𝑏𝐸^𝑇,(125) which, together with (78), implies that 𝐼_2𝑛=𝜒¹_𝑏𝑐₀,𝜒²_𝑏𝑐₀,…,𝜒^2𝑑𝑏_𝑏𝑐₀𝐸^𝑇.(126) By Lemma 12, one has rank𝐸=2𝑛.
On the other hand, it follows from (122) that 𝑧^𝑖_𝑏,𝜒^𝑠_𝑏(𝑏+1)=𝑦_𝑏^𝑖,0,𝜒^𝑠_𝑏(𝑏+1)+^2𝑑𝑏_𝑗=1𝑎_𝑖𝑗𝜒^𝑏_𝑗,𝜒^𝑠_𝑏(𝑏+1),(127) for 1≤𝑖≤2𝑛, 1≤𝑠≤2𝑑_𝑏, which, together with (78) and (2) of Theorem 27, implies that 𝐸𝐹₁=0.(128) Noting that rank𝐸=2𝑛, one has rank𝐹₁≤2𝑑_𝑏−2𝑛.(129) We now want to show rank𝐹₁≥2𝑑_𝑏−2𝑛. By (3) of Lemma 14 one has 𝐹₂∶=𝜒^𝑖_𝑏,𝜒^𝑗_𝑏(𝑏+1)_{1≤𝑖≤𝑑𝑏,1≤𝑗≤𝑑𝑏}=𝑋^𝑇₀𝒥𝑋₀^𝑇,(130) where 𝑋₀∶=(𝑥¹_𝑏(𝑐₀),𝑥²_𝑏(𝑐₀),…,𝑥^𝑑𝑏_𝑏(𝑐₀)). Since rank𝑋₀=𝑑_𝑏, we have that rank𝐹₁≥rank𝐹₂≥2𝑑_𝑏−2𝑛,(131) which, together with (129), yields that rank𝐹₁=rank𝐹₂=2𝑑_𝑏−2𝑛.(132) Because rank𝐹₂=2𝑑_𝑏−2𝑛≤𝑑_𝑏, one can rearrange the order of 𝜒¹_𝑏,𝜒²_𝑏,…,𝜒^𝑑𝑏_𝑏 such that the first 2𝑑_𝑏−2𝑛 rows of 𝐹₂ are linearly independent; that is, (124) holds. The proof is complete.
Without loss of generality, we assume that (124) holds in the rest of this paper. Now, we can give a characterization of 𝐻_𝑏(𝜏).

Theorem 37. Assume that (𝐴₁) and (𝐴_𝑏,2) hold, 𝜆∈Γ(𝐻_𝑏,0(𝜏))≠∅, and 𝜒¹_𝑏,𝜒²_𝑏,…,𝜒^2𝑑𝑏_𝑏^−2𝑛 are linearly independent solutions of 1_𝜆 in ℒ²_𝑤(ℐ_𝑏) such that (124) holds. Then 𝐺_𝑏∶=𝜒^𝑖_𝑏,𝜒^𝑗_𝑏(𝑏+1)_{1≤𝑖,𝑗≤2𝑑𝑏−2𝑛}(133) is invertible, and any 𝑦∈Dom𝐻_𝑏(𝜏) can be uniquely expressed as 𝑦=𝑦⁰_𝑏+^2𝑛_𝑖=1𝑐_𝑖𝑧^𝑖_𝑏+^{2𝑑𝑏−2𝑛}_𝑗=1𝑑_𝑗𝜒^𝑗_𝑏,(134) where 𝑦⁰_𝑏∈Dom𝐻_𝑏,0(𝜏), 𝑧^𝑖_𝑏 are defined by (78), and 𝑐_𝑖,𝑑_𝑗∈ℂ.

Proof. Let 𝐸=(𝐸₁,𝐸₂), where 𝐸₁ and 𝐸₂ are 2𝑛×(2𝑑_𝑏−2𝑛) and 2𝑛×2𝑛 matrices, respectively. It follows from (124) that there exists an invertible matrix 𝐿 such that 𝐹₁𝐼𝐿=^{2𝑑𝑏−2𝑛}0𝐹₃𝐹₄.(135) So, it follows from (128) that 𝐸₁+𝐸₂𝐹₃=0, which is equivalent to 𝐸₁=−𝐸₂𝐹₃. Since rank𝐸=2𝑛, it follows that 𝐸₂ is invertible. Multiplying (125) by (𝐸₂⁻¹)^𝑇 from the right-hand side, we get 𝑧¹_𝑏,𝑧²_𝑏,…,𝑧_𝑏^2𝑛𝐸₂^−1𝑇=𝑦_𝑏^1,0,𝑦_𝑏^2,0,…,𝑦_𝑏^2𝑛,0𝐸₂^−1𝑇+𝜒¹_𝑏,𝜒²_𝑏,…,𝜒^2𝑑𝑏_𝑏^−2𝑛𝐸₂⁻¹𝐸₁^𝑇+𝜒^2𝑑𝑏_𝑏^−2𝑛+1,…,𝜒^2𝑑𝑏_𝑏.(136) This implies that each of 𝜒^2𝑑𝑏_𝑏^−2𝑛+1,…,𝜒^2𝑑𝑏_𝑏 can be uniquely expressed as a linear combination of 𝜒¹_𝑏,𝜒²_𝑏,…,𝜒^2𝑑𝑏_𝑏^−2𝑛, 𝑦_𝑏^1,0,𝑦_𝑏^2,0,…,𝑦_𝑏^2𝑛,0, and 𝑧¹_𝑏,𝑧²_𝑏,…,𝑧_𝑏^2𝑛. Therefore, (134) follows from Lemma 35.
Since 𝜒^𝑗_𝑏∈Dom𝐻_𝑏(𝜏) for 1≤𝑗≤𝑑_𝑏, 𝜒^𝑗_𝑏 can be uniquely expressed as 𝜒^𝑗_𝑏=𝑦_𝑏^𝑗,0+^2𝑛_𝑙=1𝑏_𝑗𝑙𝑧^𝑙_𝑏+^{2𝑑𝑏−2𝑛}_𝑠=1𝑐_𝑗𝑠𝜒^𝑠_𝑏,1≤𝑗≤𝑑_𝑏,(137) where 𝑦_𝑏^𝑗,0∈Dom𝐻_𝑏,0(𝜏), and 𝑏_𝑗𝑙,𝑐_𝑗𝑠∈ℂ. This, together with (78) and (2) of Theorem 27, implies that for 1≤𝑖≤2𝑑_𝑏−2𝑛,1≤𝑗≤𝑑_𝑏𝜒^𝑖_𝑏,𝜒^𝑗_𝑏(𝑏+1)=^{2𝑑𝑏−2𝑛}_𝑠=1𝑐_𝑗𝑠𝜒^𝑖_𝑏,𝜒^𝑏_𝑠(𝑏+1),(138) that is, 𝜒^𝑖_𝑏,𝜒^𝑗_𝑏(𝑏+1)_{1≤𝑖≤2𝑑𝑏−2𝑛,1≤𝑗≤𝑑𝑏}=𝐺_𝑏𝑐_𝑖𝑗^𝑇_{1≤𝑖≤𝑑𝑏,1≤𝑗≤2𝑑𝑏−2𝑛}.(139) Therefore, 𝐺_𝑏 is invertible from (124). This completes the proof.

With a similar argument, one can obtain the following characterization of 𝐻_𝑎(𝜏).

Theorem 38. Assume that (𝐴₁) and (𝐴_𝑎,2) hold, Γ(𝐻_𝑎,0(𝜏))≠∅. Then, for some 𝜆∈Γ(𝐻_𝑎,0(𝜏)), system 1_𝜆 has 2𝑑_𝑎−2𝑛 linearly independent solutions 𝜒¹_𝑎,…,𝜒^2𝑑𝑎_𝑎^−2𝑛 in ℒ²_𝑊(ℐ_𝑎) such that 𝐺_𝑎 is invertible, where 𝐺_𝑎∶=𝜒^𝑖_𝑎,𝜒^𝑗_𝑎(𝑎)_{1≤𝑖,𝑗≤2𝑑𝑎−2𝑛},(140) and each 𝑦∈Dom𝐻_𝑎(𝜏) can be uniquely expressed as 𝑦=𝑦⁰_𝑎+^2𝑛_𝑖=1𝑐_𝑖𝑧^𝑖_𝑎+^{2𝑑𝑎−2𝑛}_𝑗=1𝑑_𝑗𝜒^𝑗_𝑎,(141) where 𝑦⁰_𝑎∈Dom𝐻_𝑎,0(𝜏), 𝑐_𝑖,𝑑_𝑗∈ℂ, and 𝑧^𝑖_𝑎 are defined as (79).

5. Characterizations of 𝐽-SSEs of 𝐻₀(𝜏)

In this section, we give a complete characterization of all the 𝐽-SSEs of minimal subspace 𝐻₀(𝜏) in terms of the square summable solutions of system 1_𝜆. As a consequence, characterizations of all the 𝐽-self-adjoint subspace extensions are obtained in the two special cases: the limit point and limit circle cases. The following discussion is divided into two parts based on the form of ℐ.

5.1. Both the Endpoints Are Infinite

Let ℐ=(−∞,+∞), and assume that (𝐴₁), (𝐴_𝑎,2), and (𝐴_𝑏,2) hold, and Γ(𝐻₀(𝜏))≠∅ in this subsection. It follows from Theorem 32 that 𝑑_𝑎+𝑑_𝑏−2𝑛=𝑑.(142) In addition, for some 𝜆∈Γ(𝐻₀(𝜏), let 𝜒¹_𝑏,…,𝜒^2𝑑𝑏_𝑏^−2𝑛 given in Theorems 37 and 𝜒¹_𝑎,…,𝜒^2𝑑𝑎_𝑎^−2𝑛 be given in Theorems 38.

Theorem 39. Assume that (𝐴₁), (𝐴_𝑎,2), (𝐴_𝑏,2), and (𝐴₃) hold. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exist two matrices 𝑀_{𝑑×(2𝑑𝑎−2𝑛)} and 𝑁_{𝑑×(2𝑑𝑏−2𝑛)} such that (1)rank(𝑀,𝑁)=𝑑, (2)𝑁𝐺_𝑏𝑁^𝑇−𝑀𝐺_𝑎𝑀^𝑇=0, and 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑀𝑦,𝜒¹_𝑎()⋮−∞𝑦,𝜒^2𝑑𝑎_𝑎^−2𝑛(−∞)−𝑁𝑦,𝜒¹_𝑏()⋮+∞𝑦,𝜒^2𝑑𝑏_𝑏^−2𝑛(),+∞=0(143)where 𝐺_𝑏 and 𝐺_𝑎 are the same as those in Theorems 37 and 38.

Proof. We first show the sufficiency.
Suppose that there exist two matrices 𝑀_{𝑑×(2𝑑𝑎−2𝑛)} and 𝑁_{𝑑×(2𝑑𝑏−2𝑛)} such that conditions (1) and (2) hold and 𝑇 is defined by (143). We now prove that 𝑇 is a 𝐽-self-adjoint subspace extension of 𝐻₀(𝜏) by Lemma 9.
Denote 𝑀=𝑚_{𝑖𝑗𝑑×(2𝑑𝑎−2𝑛)},𝑁=𝑛_{𝑖𝑗𝑑×(2𝑑𝑏−2𝑛)},(144) and set 𝑤^𝑖_𝑎∶=^{2𝑑𝑎−2𝑛}_𝑗=1𝑚_𝑖𝑗𝜒^𝑗_𝑎,𝑤^𝑖_𝑏∶=^{2𝑑𝑏−2𝑛}_𝑗=1𝑛_𝑖𝑗𝜒^𝑗_𝑏,1≤𝑖≤𝑑.(145) Clearly, 𝑤^𝑖_𝑎∈Dom𝐻_𝑎(𝜏) and 𝑤^𝑖_𝑏∈Dom𝐻_𝑏(𝜏) for 1≤𝑖≤𝑑. By Remark 26, there exist 𝛽_𝑖∶=(𝜔_𝑖,𝜌^𝜋_𝑖)∈𝐻(𝜏)(1≤𝑖≤𝑑) such that 𝜔_𝑖(𝑡)=𝑤^𝑖_𝑎(𝑡),𝑡≤𝑠⁰_𝑎,𝜔_𝑖(𝑡)=𝑤^𝑖_𝑏(𝑡),𝑡≥𝑡⁰_𝑏+1,(146) where 𝑠⁰_𝑎 and 𝑡⁰_𝑏 are specified by (𝐴_𝑎,2) and (𝐴_𝑏,2), respectively.
Since 𝐻(𝜏) and 𝐻₀(𝜏) are liner subspaces, 𝛽₁,𝛽₂,…,𝛽_𝑑 are linearly independent in 𝐻(𝜏) (modulo 𝐻₀(𝜏)) if and only if 𝜔₁,𝜔₂,…,𝜔_𝑑 are linearly independent in Dom𝐻(𝜏) (modulo Dom𝐻₀(𝜏)). So it suffices to show that 𝜔₁,𝜔₂,…,𝜔_𝑑 are linearly independent in Dom𝐻(𝜏) (modulo Dom𝐻₀(𝜏)). Suppose that there exists 𝐶=(𝑐₁,𝑐₂,…,𝑐_𝑑)∈ℂ^𝑑 such that 𝜔=^𝑑_𝑗=1𝑐_𝑗𝜔_𝑗∈Dom𝐻₀(𝜏).(147) It follows from (145), (146), and (2) and (3) of Theorem 27 that 0=𝜔,𝜒¹_𝑎(−∞),…,𝜔,𝜒^2𝑑𝑎_𝑎^−2𝑛(−∞)=𝐶𝑀𝐺_𝑎,0=𝜔,𝜒¹_𝑏(+∞),…,𝜔,𝜒_𝑏^{2𝑑−2𝑛}(+∞)=𝐶𝑁𝐺_𝑏.(148) Since 𝐺_𝑏 and 𝐺_𝑎 are invertible, we get from (148) that 𝐶𝑀=𝐶𝑁=0. Then 𝐶=0 by condition (1). So, 𝜔₁,𝜔₂,…,𝜔_𝑑 are linearly independent in Dom𝐻(𝜏) (modulo Dom𝐻₀(𝜏)), and consequently 𝛽₁,𝛽₂,…,𝛽_𝑑 are linearly independent in 𝐻(𝜏) (modulo 𝐻₀(𝜏)).
Next, we show that [𝛽_𝑖∶𝛽_𝑗]=0 for 1≤𝑖,𝑗≤𝑑. It follows from (145) and (146) that 𝜔_𝑖,𝜔_𝑗(−∞)_{1≤𝑖,𝑗≤𝑑}=𝑀𝐺_𝑎𝑀^𝑇,𝜔_𝑖,𝜔_𝑗(+∞)_{1≤𝑖,𝑗≤𝑑}=𝑁𝐺_𝑏𝑁^𝑇,(149) which, together with Lemma 24 and condition (2), implies that 𝛽_𝑖∶𝛽_{𝑗1≤𝑖,𝑗≤𝑑}=𝑁𝐺_𝑏𝑁^𝑇−𝑀𝐺_𝑎𝑀^𝑇=0.(150) Consequently, [𝛽_𝑖∶𝛽_𝑗]=0 for 1≤𝑖, 𝑗≤𝑑. Therefore, {𝛽_𝑗}^𝑑_𝑗=1 satisfy the conditions (1) and (2) of Lemma 9.
Note that for each 𝑦∈Dom𝐻(𝜏), it follows that 𝑀𝑦,𝜒¹_𝑎()⋮−∞𝑦,𝜒^2𝑑𝑎_𝑎^−2𝑛)=(−∞𝑦,∑^{2𝑑𝑎−2𝑛}_𝑗=1𝑚_1𝑗𝜒^𝑗_𝑎)⋮(−∞𝑦,∑^{2𝑑𝑎−2𝑛}_𝑗=1𝑚_𝑑𝑗𝜒^𝑗_𝑎()=−∞𝑦,𝜔₁()⋮−∞𝑦,𝜔_𝑑(),𝑁−∞𝑦,𝜒¹_𝑏()⋮+∞𝑦,𝜒_𝑏^{2𝑑−2𝑛}()=+∞𝑦,∑^{2𝑑−2𝑛}_𝑗=1𝑛_1𝑗𝜒^𝑗_𝑏)⋮(+∞𝑦,∑^{2𝑑−2𝑛}_𝑗=1𝑛_𝑑𝑗𝜒^𝑗_𝑏()=+∞𝑦,𝜔₁()⋮+∞𝑦,𝜔_𝑑(),+∞(151) where (145) and (146) have been used. Therefore, it follows from Lemma 24 that 𝑇 can be expressed as 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑦,𝑔^𝜋∶𝛽_𝑗=0,1≤𝑗≤𝑑.(152) Hence, 𝑇 is a 𝐽-SSE of 𝐻₀(𝜏) by Lemma 9. The sufficiency is proved.
We now show the necessity. Suppose that 𝑇 is a 𝐽-SSE of 𝐻₀(𝜏). By Lemma 9 and Theorem 17, there exists a set of {𝛽_𝑗}^𝑑_𝑗=1⊂𝐻(𝜏) such that (152) holds. Write 𝛽_𝑗=(𝜔_𝑗,𝜌^𝜋_𝑗). Then 𝜔^𝑗_𝑎∈Dom𝐻_𝑎(𝜏) and 𝜔^𝑗_𝑏∈Dom𝐻_𝑏(𝜏) for 1≤𝑗≤𝑑. By Theorems 37 and 38, each 𝜔^𝑗_𝑎 and 𝜔^𝑗_𝑏 can be uniquely expressed as 𝜔^𝑗_𝑎=𝑦_𝑎^𝑗,0+^2𝑛_𝑖=1𝑐_𝑖𝑗𝑧^𝑖_𝑎+^{2𝑑𝑎−2𝑛}_𝑠=1𝑚_𝑗𝑠𝜒^𝑠_𝑎,𝜔^𝑗_𝑏=𝑦_𝑏^𝑗,0+^2𝑛_𝑖=1𝑑_𝑖𝑗𝑧^𝑖_𝑏+^{2𝑑𝑏−2𝑛}_𝑠=1𝑛_𝑗𝑠𝜒^𝑠_𝑏,1≤𝑗≤𝑑,(153) where 𝑦_𝑏^𝑗,0∈Dom𝐻_𝑏,0(𝜏), 𝑦_𝑎^𝑗,0∈Dom𝐻_𝑎,0(𝜏), and 𝑑_𝑖𝑗,𝑐_𝑖𝑗,𝑛_𝑗𝑠,𝑚_𝑗𝑠∈ℂ. Set 𝑀=𝑚_{𝑗𝑠𝑑×(2𝑑𝑎−2𝑛)},𝑁=𝑛_{𝑗𝑠𝑑×(2𝑑𝑏−2𝑛)}.(154)
First, we want to show that 𝑀 and 𝑁 satisfy condition (1). Otherwise, suppose that rank(𝑀,𝑁)<𝑑. Then there exists 𝐶=(𝑐₁,𝑐₂,…,𝑐_𝑑)∈ℂ^𝑑 with 𝐶≠0 such that 𝐶(𝑀,𝑁)=0.(155) Then 𝐶𝑀=0 and 𝐶𝑁=0. Let 𝜔=∑^𝑑_𝑗=1𝑐_𝑗𝜔_𝑗. We then have 𝜔,𝜒¹_𝑎(−∞),…,𝜔,𝜒^2𝑑𝑎_𝑎^−2𝑛(−∞)=𝐶𝑀𝐺_𝑎=0𝜔,𝜒¹_𝑏(+∞),…,𝜔,𝜒_𝑏^{2𝑑−2𝑛}(+∞)=𝐶𝑁𝐺_𝑏=0.(156) For each 𝑦∈Dom𝐻(𝜏), 𝑦_𝑏 can be uniquely expressed as (134) by Theorem 37 and 𝑦_𝑎 can be uniquely expressed as (141) by Theorem 38. So it follows from (156), (78), (79), and (2) and (3) of Theorem 27 that for any 𝑦∈Dom𝐻(𝜏), (𝜔,𝑦)(+∞)=(𝜔,𝑦)(−∞)=0, which yields that 𝜔∈Dom𝐻₀(𝜏) by (1) of Theorem 27. Then 𝜔₁,𝜔₂,…,𝜔_𝑑 are linearly dependent in Dom𝐻(𝜏) (modulo Dom𝐻₀(𝜏)), and consequently 𝛽₁,𝛽₂,…,𝛽_𝑑 are linearly dependent in 𝐻(𝜏) (modulo 𝐻₀(𝜏)). This is a contradiction. Hence, rank(𝑀,𝑁)=𝑑.
Next, we prove that 𝑀 and 𝑁 satisfy condition (2). It can be easily verified that 𝜔_𝑖,𝜔_𝑗(−∞)_{1≤𝑖,𝑗≤𝑑}=𝑀𝐺_𝑎𝑀^𝑇,𝜔_𝑖,𝜔_𝑗(+∞)_{1≤𝑖,𝑗≤𝑑}=𝑁𝐺_𝑏𝑁^𝑇.(157) Hence, by Lemma 14 and [𝛽_𝑖∶𝛽_𝑗]=0 for 1≤𝑖, 𝑗≤𝑑, 𝑀 and 𝑁 satisfy condition (2).
In addition, it follows from (78), (79), (153), and (2) and (3) of Theorem 27 that 𝑦,𝜔₁()⋮−∞𝑦,𝜔_𝑑)=(−∞𝑦,∑^{2𝑑𝑎−2𝑛}_𝑗=1𝑚_1𝑗𝜒^𝑗_𝑎)⋮(−∞𝑦,∑^{2𝑑𝑎−2𝑛}_𝑗=1𝑚_𝑑𝑗𝜒^𝑗_𝑎(−∞)=𝑀𝑦,𝜒¹_𝑎()⋮−∞𝑦,𝜒^2𝑑𝑎_𝑎^−2𝑛(),−∞𝑦,𝜔₁()⋮+∞𝑦,𝜔_𝑑()=+∞𝑦,∑^{2𝑑−2𝑛}_𝑗=1𝑛_1𝑗𝜒^𝑗_𝑏)⋮(+∞𝑦,∑^{2𝑑−2𝑛}_𝑗=1𝑛_𝑑𝑗𝜒^𝑗_𝑏(+∞)=𝑁𝑦,𝜒¹_𝑏()⋮+∞𝑦,𝜒_𝑏^{2𝑑−2𝑛}().+∞(158) Hence, 𝑇 in (152) can be expressed as (143). The necessity is proved.
The entire proof is complete.

To end this subsection, we give characterizations of 𝐽-SSEs of 𝐻₀(𝜏) in four special cases of defect indices: 𝑑_𝑎=𝑑_𝑏=𝑛; 𝑑_𝑎=𝑛,𝑑_𝑏=2𝑛; 𝑑_𝑎=2𝑛,𝑑_𝑏=𝑛; 𝑑_𝑎=𝑑_𝑏=2𝑛.

In the case that 𝑑_𝑎=𝑑_𝑏=𝑛, that is, 𝑑=0 by Theorem 32, the following result is derived from Lemma 11 and Theorem 17.

Theorem 40. Assume that (𝐴₁), (𝐴_𝑎,2), (𝐴_𝑏,2), and (𝐴₃) hold. If 𝑑=0, then 𝐻₀(𝜏) is a 𝐽-self-adjoint subspace.

In the case that 𝑑_𝑎=𝑛, 𝑑_𝑏=2𝑛, it follows from Theorem 32 that 𝑑=𝑛. Let 𝜒^𝑖_𝑏, (1≤𝑖≤2𝑛), be 2𝑛 linearly independent solutions in ℒ²_𝑊(ℐ_𝑏) of 1_𝜆 satisfying (𝜒¹_𝑏(𝑐₀),𝜒²_𝑏(𝑐₀),…,𝜒_𝑏^2𝑛(𝑐₀))=𝐼_2𝑛. Then, by (3) of Lemma 14 one has that 𝐺_𝑏=𝜒^𝑖_𝑏,𝜒^𝑗_𝑏𝑐_{01≤𝑖,𝑗≤2𝑛}=𝐽₀𝑌^∗𝑐₀𝒥𝑌𝑐₀=𝒥.(159) The following result can be directly derived from Theorem 39.

Theorem 41. Assume that (𝐴₁), (𝐴₂), (𝐴_𝑎,2), (𝐴_𝑏,2), and (𝐴₃) hold. If 1_𝜆 is in l.p.c. at 𝑡=−∞ and in l.c.c. at 𝑡=+∞. Let 𝜒^𝑖_𝑏(1≤𝑖≤2𝑛) be 2𝑛 linearly independent solutions in ℒ²_𝑊(ℐ_𝑏) of 1_𝜆 satisfying (𝜒^𝑏₁(𝑐₀),𝜒^𝑏₂(𝑐₀),…,𝜒^𝑏_2𝑛(𝑐₀))=𝐼_2𝑛. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exists a matrix 𝑁_𝑛×2𝑛 such that rank𝑁=𝑛,𝑁𝒥𝑁^𝑇=0,𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑁𝑦,𝜒¹_𝑏)⋮(+∞𝑦,𝜒_𝑏^2𝑛().+∞=0(160)

In the case that 𝑑_𝑎=2𝑛,𝑑_𝑏=𝑛, a similar result can be easily given. So we omit the details in this case.

In the case that 𝑑_𝑎=𝑑_𝑏=2𝑛, it follows from Theorem 32 that 𝑑=2𝑛. Let 𝜒_𝑖(1≤𝑖≤2𝑛) be 2𝑛 linearly independent solutions in ℒ²_𝑊(ℐ) of 1_𝜆 satisfying 𝜒₁𝑐₀,𝜒₂𝑐₀,…,𝜒_2𝑛𝑐₀=𝐼_2𝑛.(161) Then, by Lemma 14, 𝐺_𝑏 and 𝐺_𝑎, defined by (133) and (140), satisfy 𝐺_𝑎=𝐺_𝑏=𝒥.(162) The following result is a direct consequence of Theorem 39.

Theorem 42. Assume that (𝐴₁), (𝐴_𝑎,2), (𝐴_𝑏,2), and (𝐴₃) hold. If 1_𝜆 is in l.c.c. at both 𝑡=+∞ and 𝑡=−∞, then for any given 𝜆∈Γ(𝐻₀(𝜏)), let 𝜒_𝑖(1≤𝑖≤2𝑛) be 2𝑛 linearly independent solutions in ℒ²_𝑊(ℐ) of 1_𝜆 satisfying (𝜒₁(𝑐₀),𝜒₂(𝑐₀),…,𝜙_2𝑛(𝑐₀))=𝐼_2𝑛. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exist two matrices 𝑀_2𝑛×2𝑛 and 𝑁_2𝑛×2𝑛 such that rank(𝑀,𝑁)=2𝑛,𝑀𝒥𝑀^𝑇=𝑁𝒥𝑁^𝑇,(163)𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑀𝑦,𝜒₁)⋮(−∞𝑦,𝜒_2𝑛(−∞)−𝑁𝑦,𝜒₁()⋮+∞𝑦,𝜒_2𝑛().+∞=0(164)

Remark 43. As we have seen, there is no boundary condition at the endpoints at which system 1_𝜆 is in l.p.c., and the matrix 𝐺_𝑎 or 𝐺_𝑏 can be replaced by 𝒥 in the case that system 1_𝜆 is in l.c.c. at 𝑡=𝑎 or 𝑡=𝑏.

5.2. At Least One of the Two Endpoints Is Finite

In this subsection, we characterize the 𝐽-SSEs of 𝐻₀(𝜏) in the special case that at least one of the two endpoints 𝑎 and 𝑏 is finite. We first consider the case that 𝑎 is finite, and 𝑏 is finite or infinite.

We point out that in this case, characterizations of all the 𝐽-SSEs of 𝐻₀(𝜏) can also be given by the proof of Theorem 39, provided that assumptions (𝐴_𝑏,2), (𝐴_𝑎,2), and (𝐴₃) hold. But, if there does not exist a 𝑐₀∈ℐ such that both (𝐴_𝑎,2) and (𝐴_𝑏,2) are satisfied, then Theorem 39 fails. We will remark again that the division of ℐ is not necessary in the case that one of the two endpoints is finite, and characterizations of all the 𝐽-SSEs of 𝐻₀(𝜏) can still be given provided that (𝐴₃) and (𝐴₃) hold.

In the case that 𝑎 is finite, ℐ can be regarded as ℐ_𝑏 with 𝑎=𝑐₀, and (𝐴₂) is equivalent to (𝐴_𝑏,2). So all the characterizations for 𝐻_𝑏,0(𝜏) and 𝐻_𝑏(𝜏) given in Sections 3 and 4 are available to 𝐻₀(𝜏) and 𝐻(𝜏), respectively, with 𝑐₀ replaced by 𝑎. Assume that (𝐴₂) holds. Then for any given 𝜆∈Γ(𝐻₀(𝜏)), as discussed in Section 4, let 𝜒₁,…,𝜒_{2𝑑−2𝑛} be 2𝑑−2𝑛 linearly independent solutions in ℒ²_𝑊(ℐ) of 1_𝜆 such that 𝐺 is invertible, where 𝐺 is defined by (133) with 𝜒^𝑗_𝑏 is replaced by 𝜒_𝑗, 1≤𝑗≤2𝑑−2𝑛. Then all the results of Theorem 37 hold with 𝐺_𝑏 and 𝜒^𝑗_𝑏 replaced by 𝐺 and 𝜒_𝑗, respectively.

Theorem 44. Assume that the left endpoint 𝑎 is finite, and (𝐴₁)--(𝐴₃) hold. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exist two matrices 𝑀_𝑑×2𝑛 and 𝑁_{𝑑×(2𝑑−2𝑛)} such that (1)rank(𝑀,𝑁)=𝑑, (2)𝑀𝒥𝑀^𝑇−𝑁𝐺𝑁^𝑇=0, and 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑀𝑦(𝑎)−𝑁𝑦,𝜒₁()⋮𝑏+1𝑦,𝜒_{2𝑑−2𝑛}().𝑏+1=0(165)

Proof. The main idea of the proof is similar to that of Theorem 39.
We first show the sufficiency. Denote 𝒥𝑀^𝑇=𝜉₁,𝜉₂,…,𝜉_𝑑,𝑁=𝑛_{𝑖𝑗𝑑×(2𝑑−2𝑛)},(166) and set 𝑤_𝑖∶=^{2𝑑−2𝑛}_𝑗=1𝑛_𝑖𝑗𝜒_𝑗,1≤𝑖≤𝑑.(167) Clearly, 𝑤_𝑖∈Dom𝐻(𝜏) for 1≤𝑖≤𝑑. By Remark 26, there exist 𝛽_𝑖∶=(𝜔_𝑖,𝜌^𝜋)∈𝐻(𝜏) (1≤𝑖≤𝑑) such that 𝜔_𝑖(𝑎)=𝜉_𝑖,𝜔_𝑖(𝑡)=𝑤_𝑖(𝑡),𝑡≥𝑡₀+1,(168) where 𝑡₀ is specified by (𝐴₂). It can be verified by the method used in the proof of Theorem 39 that the set {𝛽_𝑗}^𝑑_𝑗=1 satisfies the conditions (1) and (2) in Lemma 9. Note that for each 𝑦∈Dom𝐻(𝜏), it follows that 𝑀𝑦(𝑎)=𝒥𝑀^𝑇𝑇(𝑎)=𝜔𝒥𝑦_𝑇1(𝑎)⋮𝜔^𝑇_𝑑(𝑎)𝒥𝑦(𝑎)=𝑦,𝜔₁(𝑎)⋮𝑦,𝜔_𝑑(𝑎),𝑁𝑦,𝜒₁()⋮𝑏+1𝑦,𝜒_{2𝑑−2𝑛})=(𝑏+1𝑦,∑^{2𝑑−2𝑛}_𝑗=1𝑛_1𝑗𝜒_𝑗()⋮𝑏+1𝑦,∑^{2𝑑−2𝑛}_𝑗=1𝑛_𝑑𝑗𝜒_𝑗()=𝑏+1𝑦,𝜔₁()⋮𝑏+1𝑦,𝜔_𝑑().𝑏+1(169) Therefore, it follows from Lemma 24 that 𝑇 can be expressed as 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑦,𝑔^𝜋∶𝛽_𝑗=0,1≤𝑗≤𝑑.(170) Hence, 𝑇 is a 𝐽-SSE of 𝐻₀(𝜏) by Lemma 9. The sufficiency is proved.
We now show the necessity. Suppose that 𝑇 is a 𝐽-SSE of 𝐻₀(𝜏). By Lemma 9 and Theorem 17, there exists a set of {𝛽_𝑗}^𝑑_𝑗=1⊂𝐻(𝜏) for {𝐻₀(𝜏),𝐻(𝜏)} such that conditions (1) and (2) in Lemma 9 hold, and 𝑇 can be expressed as (152). Write 𝛽_𝑖∶=(𝜔_𝑖,𝜌^𝜋). Then 𝜔_𝑗∈Dom𝐻(𝜏) for 1≤𝑗≤𝑑. By Theorem 37, each 𝜔_𝑖 can be uniquely expressed as 𝜔_𝑖=𝑦_𝑏^𝑖,0+^2𝑛_𝑗=1𝑎_𝑖𝑗𝑧^𝑗_𝑏+^{2𝑑−2𝑛}_𝑠=1𝑛_𝑖𝑠𝜒^𝑠_𝑏,1≤𝑖≤𝑑,(171) where 𝑦_𝑏^𝑖,0∈Dom𝐻₀(𝜏) and 𝑎_𝑖𝑗,𝑛_𝑖𝑠∈ℂ. Set 𝑀_𝑑×2𝑛=𝜔₁(𝑎),…,𝜔_𝑑(𝑎)^𝑇𝑁𝒥,_{𝑑×(2𝑑−2𝑛)}=𝑛_{𝑖𝑠1≤𝑖≤𝑑,1≤𝑠≤2𝑑−2𝑛}.(172)
With a similar argument to that used in the proof of Theorem 39, we can prove that 𝑀 and 𝑁 satisfy conditions (1) and (2). In addition, it is clear that 𝑇 in (170) can be expressed as (165). The necessity is proved, and then the entire proof is complete.

At the end of this subsection, we give the characterizations of 𝐽-SSEs of 𝐻₀(𝜏) in two special cases of defect indices.

In the special case that 𝑑=𝑛, Theorem 44 can be described in the following simpler form.

Theorem 45. Assume that the left endpoint 𝑎 is finite, and (𝐴₁)--(𝐴₃) hold. If 1_𝜆 is in l.p.c. at 𝑡=𝑏, then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exists a matrix 𝑀_𝑛×2𝑛 satisfying the self-adjoint condition rank𝑀=𝑛,𝑀𝒥𝑀^𝑇=0(173) such that 𝑇 can be defined by 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑀𝑦(𝑎)=0.(174)

In the other special case that 𝑑=2𝑛, the following result is a direct consequence of Theorem 44.

Theorem 46. Assume that the left endpoint 𝑎 is finite, and (𝐴₁)--(𝐴₃) hold. If 1_𝜆 is in l.c.c. at 𝑡=𝑏, let 𝜒_𝑖(1≤𝑖≤2𝑛) be 2𝑛 linearly independent solutions in ℒ²_𝑊(ℐ) of 1_𝜆 satisfying (𝜒₁(𝑐₀),𝜒₂(𝑐₀),…,𝜒_2𝑛(𝑐₀))=𝐼_2𝑛. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exist two 2𝑛×2𝑛 matrices 𝑀 and 𝑁 such that (1)rank(𝑀,𝑁)=2𝑛, (2)𝑀𝒥𝑀^𝑇=𝑁𝒥𝑁^𝑇, and 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑀𝑦(𝑎)−𝑁𝑦,𝜒₁()⋮𝑏+1𝑦,𝜒_2𝑛().𝑏+1=0(175)

For the case that 𝑏 is finite and 𝑎=−∞, it can be considered by a similar method. Here we only give the following basic result.

Theorem 47. Assume that the right endpoint 𝑏 is finite, and (𝐴₁)--(𝐴₃) hold. Let 𝜒_𝑗, 1≤𝑗≤2𝑑−2𝑛, be linearly independent solutions of 1_𝜆 in ℒ²_𝑊(ℐ) such that 𝐺 is invertible, where 𝐺 is defined as 𝐺_𝑎 in Theorem 38 with 𝜒^𝑗_𝑎 replaced by 𝜒_𝑗. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exist two matrices 𝑀_𝑛×2𝑛 and 𝑁_{𝑑×(2𝑑−2𝑛)} such that (1)rank(𝑀,𝑁)=𝑑, (2)𝑀𝐺𝑀^𝑇−𝑁𝒥𝑁^𝑇=0, and 𝑇=𝑦,𝑔^𝜋∈𝐻(𝜏)∶𝑀𝑦,𝜒₁(𝑎)⋮𝑦,𝜒_{2𝑑−2𝑛}(𝑎)−𝑁𝑦(𝑏+1)=0.(176)

In the case that both the two endpoints 𝑎 and 𝑏 are finite, that is, ℐ=[𝑎,𝑏], it is clear that 𝑑=2𝑛 by (𝐴₂). The characterization of 𝐽-SSEs given in Theorem 44 can be simplified as follows.

Theorem 48. Let ℐ=[𝑎,𝑏]. Assume that (𝐴₁)--(𝐴₃) hold. Then a subspace 𝑇⊂(𝐿²_𝑊(ℐ))² is a 𝐽-SSE of 𝐻₀(𝜏) if and only if there exist two 2𝑛×2𝑛 matrices 𝑀 and 𝑁 such that (1)rank(𝑀,𝑁)=2𝑛, (2)𝑁𝒥𝑁^𝑇=𝑀𝒥𝑀^𝑇, and 𝑇=𝑦,𝑔^𝜋(𝜏)(𝑎)().∈𝐻∶𝑀𝑦−𝑁𝑦𝑏+1=0(177)

Proof. Let 𝜒_𝑗∈ℒ²_𝑊(ℐ), 1≤𝑗≤2𝑛, be defined as those in Theorem 46. Then it follows that 𝑁𝑦,𝜒₁()⋮𝑏+1𝑦,𝜒_2𝑛)𝜒(𝑏+1=𝑁^𝑇1()⋮𝜒𝑏+1_𝑇2𝑛()()𝑏+1𝒥𝑦𝑏+1=𝑁₁𝑦(𝑏+1),(178) where 𝑁₁=𝑁(𝜒₁,…,𝜒_2𝑛)^𝑇(𝑏+1)𝒥. It is evident (𝜒₁,…,𝜒_2𝑛)(𝑏+1)=𝐼_2𝑛. So by Lemma 14, one has that rank(𝑀,𝑁)=rank𝑀,𝑁₁𝑁=2𝑛,₁𝒥𝑁^𝑇₁=𝑁𝒥𝑁^𝑇.(179) Hence, the assertion follows from Theorem 46. This completes the proof.

Acknowledgment

This research was supported by the NNSF of China (Grants 11071143, 11101241, and 11226160).