Abstract
Greedy expansions with prescribed coefficients, which have been studied by V. N. Temlyakov in Banach spaces, are considered here in a narrower case of Hilbert spaces. We show that in this case the positive result on the convergence does not require monotonicity of coefficient sequence . Furthermore, we show that the condition sufficient for the convergence, namely, the inclusion , can not be relaxed at least in the power scale. At the same time, in finite-dimensional spaces, the condition can be replaced by convergence of to zero.
1. Introduction
Expansion in Fourier series [1] is a classical and comprehensively studied tool of theoretical and applied mathematics which takes an expanded function as an input and constructs a sequence of its expansion coefficients. Greedy expansions [2, 3], which are equivalent in the simplest case to Fourier series reordered by decreasing norms of terms and known in statistics and signal processing as Projection Pursuit Regression [4, 5] and Matching Pursuit [6], respectively, perform parallel computation of expansion coefficients and selection of expansion elements from a predefined dictionary. V. N. Temlyakov [3, 7] (see also [8]) proposed a type of a greedy expansion that performs only selection of expansion elements, while coefficients are prescribed in advance. The definition proposed by V. N. Temlyakov for the case of Banach spaces, in the case of Hilbert spaces, takes the following form.
Definition 1. Let be a Hilbert space over with a scalar product , be a symmetric unit-normed dictionary in (i.e., , all elements in have a unit norm, and if , then also belongs to ). In addition, let , , and be a sequence of positive numbers. We define inductively a sequence of remainders and a sequence of expanding elements . First, we set . Then, if has already been defined, we select as an (arbitrary) element which satisfies the condition and set .
The series is called a greedy expansion of in the dictionary with the prescribed coefficients and the weakness parameter .
It immediately follows from the definition of a greedy expansion that , and hence the convergence of the expansion to an expanded element is equivalent to the convergence of remainders to zero as .
As a selection of an expanding element is potentially not unique, there may exist different realizations of a greedy expansion for a given dictionary , weakness parameter and sequence of coefficients . Furthermore, for greedy expansion may turn out to be nonrealizable due to the absence of an element which provides .
V. N. Temlyakov showed [3, heorem 2.1] that if a number of conditions hold which are equivalent in the case of Hilbert spaces to the divergence of the series and the convergence of the series , a greedy expansion with prescribed coefficients converges to an expanded element at least for a subsequence of indexes, i.e., . Later V. N. Temlyakov proved the standard convergence (i.e., ) under the additional condition of monotonicity of [7, heorem 4]. Yet, it remained unknown whether the condition and the monotonicity condition could be essentially relaxed without violating the guaranteed convergence to an expanded element.
2. Main Results
We start with a positive result which states that in Hilbert spaces the monotonicity is not required for the standard convergence. Namely, the following theorem holds.
Theorem 2. Let be a Hilbert space, be a symmetric unit-normed dictionary in , , be a sequence of positive numbers which satisfies the conditions (i.e., ). Then for every element all realizations of its greedy expansion in the dictionary with the prescribed coefficients and the weakness parameter converge to .
It is clear that if the first condition on is violated, then there is no convergence to an expanded element for all with the norm exceeding the sum . The significance of the second condition on follows from the following theorem.
Theorem 3. There exist a Hilbert space , a symmetric unit-normed dictionary , an element and a sequence of positive numbers such that but a greedy expansion of in the dictionary with the prescribed coefficients and the weakness parameter does not converge to .
As for the second condition of Theorem 2 a boundary in the power scale is , Theorem 3 in fact shows that this condition in Theorem 2 can not be relaxed at least in the power scale.
However, the question about a possibility of a more delicate relaxation of the condition remains open. This question can be stated as follows: is it true that for every sequence of positive numbers that converges to zero but does not belong to there exist a Hilbert space , a symmetric unit-normed dictionary and an expanded element such that at least one realization of greedy expansion of in with the prescribed coefficients (and, e.g., the weakness parameter ) does not converge to ?
We note that assertions similar to Theorems 2 and 3 have been announced by O. Rassudova in her conference talk [9], but the proofs have not been published. To the best of our knowledge, in her proof of an analogue of Theorem 3 O. Rassudova used a modification of the construction [10, heorem 3] which is based on analytical estimates and does not have a clear geometric interpretation. The construction presented in our work is geometrically demonstrative.
We also note that at least for for the natural class of monotonic coefficients in case of finite-dimensional Hilbert spaces the condition in Theorem 2 can be replaced by an essentially weaker condition . The proof of this fact is presented in section The case of finite-dimensional spaces.
3. Proof of Theorem 2
The theorem can be easily derived from the equality , which holds due to the aforementioned result by V. N. Temlyakov [3, heorem 2.1]. From the definition of the greedy expansion it immediately follows thatAs coefficients are positive and the dictionary is symmetric, . Hence and thusThe condition implies that Due to the equality we have thatFrom two last assertions we obtain that for every there exists such that the following two conditions simultaneously hold: Thus using estimate (6) we get that for all . But according to the definition of the limit it directly means that . The proof of Theorem 2 is complete.
We note that for monotonic coefficients and the weakness parameter the statement of Theorem 2, which is covered in this case by [7, heorem 4], can be also derived as a corollary of the same result by V. N. Temlyakov about the convergence for a subsequence of indexes [3, heorem 2.1] and the following lemma. We find this lemma to be interesting on its own.
Lemma 4. Let and monotonically converge to zero as . Then for every realization of greedy expansion with the prescribed coefficients the sequence of norms of its remainders converges.
We begin our proof of this lemma with an estimate of a possible increase of the remainder norms. Let denote the set of all indexes for which . If the set is finite, then starting from a certain index the sequence is monotonic and thus convergent. Hence it remains to consider the case of countably infinite .
For the sake of brevity we denote the scalar products by . It follows from the definition of the greedy expansion and the symmetric property of the dictionary that all are nonnegative.
The Pythagorean theorem implies that for all indexes the equalityholds. If , then , and thusConsequentlyAt the same timeIt means that if the remainder norm increased at the transition from to , then the increase of the norm square does not exceed , for the next expansion step the increase is impossible, and the joint increase of the square of remainder norm for two steps of the expansion does not exceed and hence does not exceed , where can be set to .
Having this estimate, let us complete the proof. We note that the series converges: it can be easily derived either from the Leibniz’s alternating series test or from the inequalityLet us fix an arbitrary positive and find an index such that and simultaneously . Next we find an index such that , where denotes the infimum of the remainder norms . Then for every we have that Hence () and consequently . The proof of Lemma 4 is complete.
4. Proof of Theorem 3
Our proof of Theorem 3 includes the following blocks: construction of a dictionary with simultaneous construction of coefficients ; description of realization of greedy expansion; proof of the absence of convergence to the expanded element; obtaining the required estimate of . As a Hilbert space we take an arbitrary infinite-dimensional separable space, e.g., .
Figure 1 illustrates certain steps of the proof.
4.1. Description of the Construction
We first present the structure of the example; i.e., we describe the construction of dictionary elements and coefficients . As a part of this construction we also define the sequence of vectors (remainders) , including the expanded element .
Let be an arbitrary non-zero element of with , . We define dictionary elements and as arbitrary (unequal) unit vectors such that , and lie in one plane and the angle between and equals the angle between and and belongs to the interval .
Next, we set dictionary element to an arbitrary unit vector with the following two properties: its orthogonal projection on the plane lies on the line with directing vector , and the angle between this vector and also equals . We also find the coefficient such that for scalar products are equal or, equivalently, angles between and the vectors , , are equal. We denote this angle by . Note that : the formal justification of this fact can be based, e.g., on the equality , which directly follows from the spherical law of cosines.
Similarly, we set to an arbitrary unit vector with an orthogonal projection on the subspace lying on the line with directing vector and the angle between this vector and equal , and select the coefficient in such a way that for scalar products , , , are equal or, equivalently, angles between and the vectors , , , are equal. We denote this angle by . Again it is easy to see that , as due to the spherical law of cosines .
We continue the construction inductively. Namely, after constructing , , and we set to an arbitrary unit vector with an orthogonal projection on the subspace lying on the line with directing vector and angle between this vector and equal , and select the coefficient in such a way that for scalar products , , , , , are equal or, equivalently, angles between and the vectors , , , , , , are equal. We denote this angle by and note that as due to the spherical law of cosines, .
Let us justify formally that it is possible to find and with the required properties. Let denote an arbitrary unit vector orthogonal to the subspace (which also contains vector ), and denote the set of vectors . As we can take any of two unit vectors from the plane that have an angle with equal to . Let ; note that as the angle is acute. By construction for all the scalar product equals . For the sake of brevity we denote by . Hence for all , as , scalar products of and are the same and equal . Consequently, for an arbitrary positive and every we have equalities , . Thus, it remains to set to the solution of the linear equation , i.e., to .
4.2. Realization of Greedy Expansion
We note that a possible realization of greedy expansion in the dictionary with the prescribed coefficients (and the weakness parameter ) is a realization in which is selected as an expanding element at the -th step, and hence -th remainder coincides with . Indeed, an angle between a vector and its orthogonal projection on a subspace does not exceed an angle between this vector and any non-zero vector from the subspace. Thus while for every and every we have an equality and for the scalar product is positive, for we have an inequality
4.3. Absence of Convergence
In this subsection we show that the greedy expansion does not converge to or, equivalently, remainders do not converge to zero.
First we find the limit of . It exists as is a nondecreasing sequence with all values belonging to the interval . As noted above, for all the equality holds. Consequently, if denotes the limit , then . It implies that .
Now we show that . Indeed,ThusAs , and hence .
The law of sines gives the equalities ConsequentlyTaking into consideration the convergence , we derive from this equality thatso the absence of convergence to the expanded element is proved.
4.4. Estimate of
Here we first show that . Applying again the law of sines, but this time for the other pairs of angles, we get equalitiesLet us use equality (23):Taking into account the monotonicity of , it implies the inequality Consequently, as the series diverges, the series also diverges.
Then we show that . The equality (26) and monotonicity of imply that It remains to establish the inequality : as , it directly gives the required upper estimate of . For this inequality holds because . Next, from equality (20), taking into account that the minimum value of the function on the positive semi-axis equals , we obtain that . Besides, is an increasing function on , and the justification of the inequality can be completed by induction: Justification of this inequality completes the proof of the upper estimate of and in total the proof Theorem 3.
We note that the construction described in the proof of Theorem 3 can be straightforwardly adapted from the case of greedy expansions with prescribed coefficients to the generalized Approximate Weak Greedy Algorithm (see [10, heorem 3]).
5. The Case of Finite-Dimensional Spaces
In this section we prove the following theorem.
Theorem 5. Let the space be finite-dimensional, and let the coefficients converge to zero monotonically and satisfy the condition . Then for every symmetric unit-normed dictionary and any all realizations of greedy expansion of in with the prescribed coefficients and the weakness parameter converge to .
Thus in the finite-dimensional case at least for in the natural class of monotonic coefficient sequences conditions sufficient for the convergence of a greedy expansion to an expanded element include only the convergence of coefficients to zero and infinity of their sum. Clearly, convergence of coefficients to zero is also a necessary condition for the convergence of greedy expansion.
As the first step of the proof we note that due to Lemma 4 there exists a limit of norms of remainders of a greedy expansion with prescribed coefficients . Let us suppose that this limit is non-zero. Then starting from a certain index remainder norms are separated from zero. Due to the compactness of a sphere in a finite-dimensional space and the completeness and the symmetric property of the dictionary it implies that scalar productsare also separated from zero. In other words, there exist a positive number and an index such that for all the inequality holds. In addition there exists such an index that for all coefficient does not exceed . But consequently for it follows from equality (10) that HenceThis contradiction completes the proof of Theorem 5.
6. Conclusion
The main results of the paper state that in Hilbert spaces a greedy expansion with prescribed coefficients converges to an expanded element if coefficients satisfy certain relatively weak conditions that do not include monotonicity, and these conditions can not be essentially relaxed. At the same time we showed that for the finite-dimensional case the relaxation is possible.
Data Availability
No data were used to support this study.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this article.
Acknowledgments
We thank Prof. T.P. Lukashenko and Dr. E.D. Livshits for valuable comments and discussions. The research was supported by the Russian Federation Government Grant No. 14.W03.31.0031.