Interpolation and Best Approximation for Spherical Radial Basis Function Networks

Abstract

Within the conventional framework of a native space structure, a smooth kernel generates a small native space, and radial basis functions stemming from the smooth kernel are intended to approximate only functions from this small native space. In this paper, we embed the smooth radial basis functions in a larger native space generated by a less smooth kernel and use them to interpolate the samples. Our result shows that there exists a linear combination of spherical radial basis functions that can both exactly interpolate samples generated by functions in the larger native space and near best approximate the target function.

1. Introduction

Many scientific questions boil down to synthesizing an unknown but definite function from finitely many samples (𝑥_𝑖,𝑦_𝑖)^𝑚_𝑖=1. The purpose is to find a functional model that can effectively represent or approximate the underlying relation between the input 𝑥_𝑖 and the output 𝑦_𝑖. If the unknown functions are defined on spherical domains, then data collected by satellites or ground stations are usually not restricted on any regular region and are scattered. Thus, any numerical method relying on the structure of a “grid” is doomed to fail.

The success of the radial basis function networks methodology in Euclidean space derives from its ability to generate estimators from data with essentially unstructured geometry. Therefore, it is natural to borrow this ideal to deal with spherical scattered data. This method, called the spherical radial basis function networks (SRBFNs), has been extensively used in gravitational phenomenon [1, 2], image processing [3, 4], and learning theory [5].

Mathematically, the SRBFN can be represented as 𝑆_𝑛(𝑥)∶=^𝑛_𝑖=1𝑐_𝑖𝜙𝜉_𝑖⋅𝑥∶=^𝑛_𝑖=1𝑐_𝑖𝜙_𝜉𝑖(𝑥),𝑥∈𝐒^𝑑, (1) where 𝑐_𝑖∈𝐑, 𝜉_𝑖∈𝐒^𝑑, and 𝜙∶[−1,1]→𝐑 are called the connection weight, center, and activation function in the terminology of neural networks, respectively. Here and hereafter, 𝐒^𝑑 denotes the unit sphere embedded in the 𝑑+1-dimensional Euclidean space, 𝐑^𝑑+1. Both the connection weights 𝑐_𝑖 and the centers 𝜉_𝑖 are adjustable in the process of training. We denote by Φ_𝑛 the collection of functions formed as (1).

A basic and classical approach for training SRBFN is to construct exact interpolant based on the given samples (𝑥_𝑖,𝑦_𝑖)^𝑚, that is, to find a function 𝑆_𝑛 in Φ_𝑛 such that 𝑆_𝑛𝑥_𝑖=𝑦_𝑖,𝑖=1,…,𝑚. (2)

If the activation function 𝜙 is chosen to be positive definite [2], then the matrix 𝐴∶=(𝜙(𝑥_𝑖⋅𝑥_𝑗))^𝑚_𝑖,𝑗=1 is nonsingular. Thus, the system of (2) can be easily solved by taking the scattered data as the centers of SRBFN. This method has been already named the spherical basis function (SBF) method. Under this circumstance, the connection weights are determined via 𝐜=𝐴⁻¹𝐲, (3) where 𝐜=(𝑐₁,𝑐₂,…,𝑐_𝑚)^𝑇, 𝐲=(𝑦₁,…,𝑦_𝑚)^𝑇, 𝑈^𝑇 denotes the transpose of the matrix (or vector) 𝑈, and 𝐴⁻¹ denotes the inverse matrix of 𝐴.

For the SBF method, if the target function 𝑓 belongs to the native space 𝑁_𝜙 associated with the activation function 𝜙, then the best SBF approximant of 𝑓 is characterized by the exact interpolation: 𝑓−𝑐_𝑖^𝑚_𝑖=1𝜙_{𝑥𝑖𝑁𝜙}=inf_{𝑔∈Φ𝑚}𝑓−𝑔_𝑁𝜙, (4) where {𝑐_𝑖}^𝑚_𝑖=1 are determined by (3). This property makes the SBF interpolation strategy popular in spherical scattered data fitting [6–14]. However, there are also two disadvantages for the SBF method. On one hand, since the centers of SRBFN interpolants are chosen as the scattered data, the interpolation capability depends heavily on their geometric distributions. This implies that we cannot obtain a satisfactory interpolation error estimate if the data are “badly” located on the sphere. On the other hand, the well known “native space barrier" [11, 12, 15] shows that (4) only holds for a small class of smooth functions if 𝜙 is smooth. Therefore, for functions outside 𝑁_𝜙, the SBF interpolants are not guaranteed to be the best approximants.

Along the flavor of the previous papers [7, 11, 12], we use SRBFNs to interpolate functions in a large native space 𝑁_𝜓 associated with the kernel 𝜓 which is less smooth than 𝜙 and study its interpolation capability. Different from the previous work, the centers are chosen in advance to be quasi-uniform located on spheres, which makes the interpolation error depend on the number rather than the geometric distribution of centers. Our purpose is not to give the detailed error estimate of the SRBFN interpolation. Instead, we focus on investigating the relation between interpolation and approximation for SRBFN. Indeed, we find that there exists an SRBFN interpolant which is also the near best approximant of functions in 𝑁_𝜓, when the number of centers and geometric distribution of the scattered data satisfy a certain assumption. That is, for a suitable choice of 𝑛, there exists a function 𝑔_𝑛∈Φ_𝑛 such that(1)𝑔 exactly interpolates the samples (𝑥_𝑖,𝑦_𝑖)^𝑚_𝑖=1;(2)‖𝑓−𝑔_𝑛‖_𝑁𝜓≤𝐶inf_{𝑔∈Φ𝑛}‖𝑓−𝑔‖_𝑁𝜓,

where 𝐶 is a constant depending only on 𝑑, 𝛼, and 𝛽.

2. Positive Radial Basis Function on the Sphere

It is easy to deduce that the volume of 𝐒^𝑑, Ω_𝑑, satisfies Ω_𝑑∶=_𝐒𝑑𝑑𝜔=2𝜋^(𝑑+1)/2Γ)),((𝑑+1/2 (5) where 𝑑𝜔 denotes the surface area element on 𝐒^𝑑. For integer 𝑘≥0, the restriction to 𝐒^𝑑 of a homogeneous harmonic polynomial of degree 𝑘 on the unit sphere is called a spherical harmonic of degree 𝑘. The span of all spherical harmonics of degree 𝑘 is denoted by 𝐇^𝑑_𝑘, and the class of all spherical harmonics (or spherical polynomials) of degree 𝑘≤𝑠 is denoted by Π^𝑑_𝑠. It is obvious that Π^𝑑_𝑠=⊕^𝑠_𝑘=0𝐇^𝑑_𝑘. The dimension of 𝐇^𝑑_𝑘 is given by 𝐷^𝑑_𝑘∶=dim𝐇^𝑑_𝑘=2𝑘+𝑑−1𝑘+𝑑−1𝑘𝑘+𝑑−1,𝑘≥1;1,𝑘=0, (6) and that of Π^𝑑_𝑠 is ∑^𝑠_𝑘=0𝐷^𝑑_𝑘=𝐷_𝑠^𝑑+1∼𝑠^𝑑.

Denote by {𝑌_𝑘,𝑗}^𝐷𝑑𝑘_𝑗=1 an orthonormal basis of 𝐇^𝑑_𝑘; then the following addition formula [16, 17] holds ^𝐷𝑑𝑘_𝑙=1𝑌_𝑘,𝑙(𝑥)𝑌_𝑘,𝑙𝑦=𝐷^𝑑_𝑘Ω_𝑑𝑃_𝑘^𝑑+1𝑥⋅𝑦, (7) where 𝑃_𝑘^𝑑+1 is the Legendre polynomial with degree 𝑘 and dimension 𝑑+1. The Legendre polynomial 𝑃_𝑘^𝑑+1 can be normalized such that 𝑃_𝑘^𝑑+1(1)=1 and satisfies the orthogonality relation ¹₋₁𝑃_𝑘^𝑑+1(𝑡)𝑃_𝑗^𝑑+1(𝑡)1−𝑡^{2(𝑑−2)/2}Ω𝑑𝑡=^𝑑Ω_𝑑−1𝐷^𝑑_𝑘𝛿_𝑘,𝑗, (8) where 𝛿_𝑘,𝑗 is the usual Kronecker symbol.

Positive definite radial basis functions on spheres were introduced and characterized by Schoenberg [18]. Namely, a radial basis function 𝜑 is positive definite if and only if its expansion 𝜑𝑥⋅𝑦=^∞_𝑘=0𝑒_𝑘𝐷^𝑑_𝑘Ω_𝑑𝑃^𝑑_𝑘𝑥⋅𝑦 (9) has all Fourier-Legendre coefficients 𝑒_𝑘≥0 and ∑^∞_𝑘=0𝑒_𝑘(𝐷^𝑑_𝑘/Ω_𝑑)<∞. We define the native space 𝑁_𝜑 as 𝑁_𝜑∶=𝑓(𝑥)=^∞𝐷_𝑘=0^𝑑𝑗_𝑗=1𝑓_𝑘,𝑗𝑌_𝑘,𝑗(𝑥)∶^∞_𝑘=0𝑒_𝑘^{𝐷−1𝑑𝑗}_𝑗=1𝑓²_𝑘,𝑗<∞, (10) with its inner product 𝑓,𝑔_𝑁𝜑∶=^∞_𝑘=0𝑒_𝑘^{𝐷−1𝑑𝑗}_𝑗=1𝑓_𝑘,𝑗𝑔_𝑘,𝑗, (11) where 𝑓_𝑘,𝑗∶=∫_𝐒𝑑𝑓(𝑥)𝑌_𝑘,𝑗(𝑥)𝑑𝜔(𝑥).

3. Interpolation and Near Best Approximation

Let Λ∶={𝜂_𝑗}^𝑛_𝑗=1⊂𝐒^𝑑 be a set of points and 𝑑(𝑥,𝑦)=arccos𝑥⋅𝑦 be the spherical distance between 𝑥and 𝑦. We denote by ℎ_Λ∶=max_𝑥∈𝐒^𝑑min_𝑗𝑑(𝑥,𝜂_𝑗), 𝑞_Λ∶=(1/2)min_𝑗≠𝑘𝑑(𝜉_𝑗,𝜂_𝑘), and 𝜏_Λ∶=ℎ_Λ/𝑞_Λ the mesh norm, separation radius, and mesh ratio of Λ, respectively. It is easy to check that these three quantities describe the geometric distribution of points in Λ. The 𝜏-uniform set 𝐹_𝜏∶=𝐹_𝜏(𝐒^𝑑) is defined by the family of all centers sets Ξ with 𝜏_Λ≤𝜏.

Let 𝜙 and 𝜓 satisfy 𝜙𝑥⋅𝑦∶=^∞_𝑘=0𝑎_𝑘𝐷^𝑑_𝑘Ω_𝑑𝑃^𝑑_𝑘,𝜓𝑥⋅𝑦𝑥⋅𝑦∶=^∞_𝑘=0𝑏_𝑘𝐷^𝑑_𝑘Ω_𝑑𝑃^𝑑_𝑘𝑥⋅𝑦 (12)(13) with 𝑎_𝑘∼1+𝜆_𝑘^−𝛼,𝑏_𝑘∼1+𝜆_𝑘^−𝛽. (14)

The Sobolev embedding theorem [12] implies that if 𝛼,𝛽>𝑑/2, then 𝑁_𝜙 and 𝑁_𝜓 are continuously embedded in 𝐶(𝐒^𝑑), and so there are reproducing kernel Hilbert spaces, with reproducing kernels being 𝜙 and 𝜓, respectively.

The aim of this section is to study the relation between the exact interpolation and best approximation for Φ_𝑛 with its centers set Ξ_𝑛∶={𝜉_𝑖}^𝑛_𝑖=1 and activation function 𝜙 satisfying (12) and (14). It is obvious that such a Φ_𝑛 is a linear space. The following Theorem 1 shows that there exists an SRBFN interpolant which can near best approximate 𝑓∈𝑁_𝜓 in the metric of 𝑁_𝜓, where 𝜓 satisfies (13) and (14).

Theorem 1. Let 𝑋∶={𝑥_𝑖}^𝑚_𝑖=1 be the set of scattered data with separation radius 𝑞_𝑋, and 𝛼>𝛽>𝑑/2. If Ξ_𝑛∈𝐹_𝜏 and 𝑛≥𝐶_𝜏𝑞_𝑋^−𝑑, where 𝐶_𝜏>1 is a constant depending only on 𝜏 and 𝑑, then, for every 𝑓∈𝑁_𝜓, there exists an SRBFN interpolant 𝑆_𝑛∈Φ_𝑛 such that(i)𝑆_𝑛 exactly interpolates the samples (𝑥_𝑖,𝑦_𝑖)^𝑚_𝑖=1, (ii)‖𝑓−𝑆_𝑛‖_𝑁𝜓≤5inf_{𝑔∈Φ𝑛}‖𝑓−𝑔‖_𝑁𝜓.

Remark 2. Similar results have been considered for spherical polynomials both in 𝐶(𝐒^𝑑) and 𝑁_𝜓. Narcowich et al. [11, 12] proved that there exists a spherical polynomial interpolant of degree at most 𝐿≥𝐶𝑞_𝑋 which can also best approximate the target both in 𝐶(𝐒^𝑑) and 𝑁_𝜓.

To prove Theorem 1, we need the following three lemmas, which can be found in [12, Proposition 5.2], [12, Theorem 5.5], and [19, Example 2.10], respectively.

Lemma 3. Let 𝒴 be a (possibly complex) Banach space, 𝒱 a subspace of 𝒴, and 𝑍^∗ a finite-dimensional subspace of 𝒴^∗, the dual of 𝒴. If for every 𝑧^∗∈𝑍^∗ and some 𝛾>1, 𝛾 independent of 𝑧^∗, 𝑧^∗_𝒴∗≤𝛾𝑧^∗∣_𝒱𝒱∗, (15) then for 𝑦∈𝒴 there exists 𝑣∈𝒱 such that 𝑣 interpolates 𝑦 on 𝑍^∗; that is, 𝑧^∗(𝑦)=𝑧^∗(𝑣) for all 𝑧^∗∈𝑍^∗. In addition, 𝑣 approximates 𝑦 in the sense that ‖𝑦−𝑣‖_𝒴≤(1+2𝛾)dist_𝒴(𝑦,𝒱).

Lemma 4. Let 𝛽>𝑑/2. If 𝑓∈𝑁_𝜙, then there is a 𝑢∈Φ_𝑛 such that 𝑓−𝑢_𝑁𝜓≤𝐶ℎ_Ξ^𝛼−𝛽_𝑛𝑓_𝑁𝜙. (16)

Lemma 5. Let 𝜓 be defined in (13) and (14) and 𝛽>𝑑/2. Then for arbitrary set of real numbers {𝑐_𝑖}^𝑚_𝑖=1, we have ^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖²_𝑁𝜓≥𝐶𝑞_𝑋^𝑚2𝛽_𝑖=1𝑐²_𝑖, (17) where 𝐶 is a constant depending only on 𝑑 and 𝛽.

Now we provide the proof of Theorem 1.

Proof of Theorem 1. We apply Lemma 3 to the case in which the underlying space is the native space 𝒴=𝑁_𝜓. Let 𝑍^∗=span{𝛿_𝑥𝑖}^𝑚_𝑖=1 and 𝒱=Φ_𝑛. So in order to prove Theorem 1, it suffices to prove that for arbitrary 𝑚 real numbers {𝑐_𝑖}^𝑚_𝑖=1 such that ^𝑚_𝑖=1𝑐_𝑖𝛿_{𝑥𝑖𝑁∗𝜓}≤5^𝑚_𝑖=1𝑐_𝑖𝛿_𝑥𝑖|_{Φ𝑛Φ∗𝑛}. (18) Since 𝑁_𝜓 is a reproducing kernel Hilbert space with 𝜓 its reproducing kernel, we have ^𝑚_𝑖=1𝑐_𝑖𝛿_{𝑥𝑖𝑁∗𝜓}=sup_{𝑓∈𝑁𝜓,‖𝑓‖𝑁𝜓=1}^𝑚_𝑖=1𝑐_𝑖𝑓𝑥_𝑖=sup_{𝑓∈𝑁𝜓,‖𝑓‖𝑁𝜓=1}^𝑚_𝑖=1𝑐_𝑖𝑓,𝜓_{𝑥𝑖𝑁𝜓}=sup_{𝑓∈𝑁𝜓,‖𝑓‖𝑁𝜓=1}^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖,𝑓_𝑁𝜓=^𝑚_𝑖=1𝑐_𝑖𝜓_{𝑥𝑖𝑁𝜓}. (19) Similarly, we obtain ^𝑚_𝑖=1𝑐_𝑖𝛿_𝑥𝑖|_{Φ𝑛Φ∗𝑛}=sup_{𝑔∈Φ𝑛,‖𝑔‖𝑁𝜓=1}^𝑚_𝑖=1𝑐_𝑖𝑔𝑥_𝑖=sup_{𝑔∈Φ𝑛,‖𝑔‖𝑁𝜓=1}^𝑚_𝑖=1𝑐_𝑖𝑔,𝜓_{𝑥𝑖𝑁𝜓}=sup_{𝑔∈Φ𝑛,‖𝑔‖𝑁𝜓=1}^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖,𝑔_𝑁𝜓=‖𝑠‖_𝑁𝜓, (20) where 𝑠 is the orthogonal projection of ∑^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖 to Φ_𝑛 in the metric of 𝑁_𝜓. Then the Pythagorean theorem yields that ‖𝑠‖²_𝑁𝜓=^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖²_𝑁𝜓−^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖−𝑠²_𝑁𝜓. (21) Thus, Lemma 3 with 𝛾=2 yields that in order to prove (18), it suffices to prove ∑^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖−𝑠_𝑁𝜓∑^𝑚_𝑖=1𝑐_𝑖𝜓_{𝑥𝑖𝑁𝜓}≤√32. (22) Let 𝐿∈ℕ, 𝑃∈Π_𝐿 be the best polynomial approximation of ∑^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖 in the metric of 𝑁_𝜓; then for arbitrary 𝑠^′∈Φ_𝑛, we have ^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖−𝑠_𝑁𝜓≤^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖−𝑃_𝑁𝜓+𝑃−𝑠^′_𝑁𝜓. (23) It follows from [12, Page 382] that there exists a constant, 𝜅, depending only on 𝛽 such that for arbitrary 𝐿≥𝜅𝑞_𝑋⁻¹, there holds ^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖−𝑃_𝑁𝜓≤√34^𝑚_𝑖=1𝑐_𝑖𝜓_{𝑥𝑖𝑁𝜓}. (24)
Let 𝑠^′ be the best Φ_𝑛 approximation of 𝑃 in the metric of 𝑁_𝜓. Then it follows from Lemma 4 and the well-known Bernstein inequality [17] that 𝑃−𝑠^′_𝑁𝜓≤𝐶ℎ_Ξ^𝛼−𝛽‖𝑃‖_𝑁𝜓≤𝐶ℎ_Ξ^𝛼−𝛽𝐿^𝛼‖𝑃‖₂. (25) Since 𝑃 is the best polynomial approximation of ∑^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖 in the metric of 𝑁_𝜓, a simple computation yields ‖𝑃‖²₂=^𝑚_𝑖=1𝑐²_𝑖. (26) Furthermore, it follows from Lemma 5 that ^𝑚_𝑖=1𝑐_𝑖𝜓_𝑥𝑖²_𝑁𝜓≥𝐶𝑞_𝑋^𝑚2𝛽_𝑖=1𝑐²_𝑖. (27) Then, Ξ_𝑛∈𝐹_𝜏 together with 𝐿=max{[𝜅𝑞_𝑋⁻¹]+1,[3^𝛽/2/(4𝐶)^𝛽𝑞_𝑋]} yields that 𝑃−𝑠^′_𝑁𝜓≤𝐶ℎ_Ξ^𝛼−𝛽𝐿^𝛼𝑞_𝑋^−𝛽𝑚_𝑖=1𝑐_𝑖𝜓_{𝑥𝑖𝑁𝜓}≤𝐶𝑛^−1/𝑑𝑞_𝑋^{𝛼−𝛽𝑚}_𝑖=1𝑐_𝑖𝜓_{𝑥𝑖𝑁𝜓}, (28) where 𝐶 is a constant depending only on 𝛽, 𝑑, and 𝜏. Thus, there exists a constant 𝐶_𝜏 depending only on 𝛼, 𝛽, 𝑑, and 𝜏 such that for arbitrary 𝑛≥𝐶_𝜏𝑞^𝑑_𝑋, there holds 𝑃−𝑠^′_𝑁𝜓≤√34^𝑚_𝑖=1𝑐_𝑖𝜓_{𝑥𝑖𝑁𝜓}. (29) Inserting (24) and (29) into (23), we finish the proof of (22) and then complete the proof of Theorem 1.

Acknowledgments

An anonymous referee has carefully read the paper and has provided to us numerous constructive suggestions. As a result, the overall quality of the paper has been noticeably enhanced, to which we feel much indebted and are grateful. The research was supported by the National 973 Programming (2013CB329404), the Key Program of National Natural Science Foundation of China (Grant no. 11131006), and the National Natural Science Foundations of China (Grant no. 61075054).