Effective Space Usage Estimation for Sliding-Window Skybands

Abstract

Skyline query computes all the “best” elements which are not dominated by any other elements and thus is very important for decision-making applications. Recently, it is generalized to skyband query and a k-skyband query returns those elements dominated by no more than k, of other elements. To incorporate the skyband operator into the stream engine for monitoring skybands over sliding windows, space usage estimation for skyband operator becomes a critical issue in the query optimizer. In this paper, we firstly introduce the skyband sketch as the cost model. Based on the cost model, we propose an approach for estimating the space usage of skyband operator over sliding windows of data streams under the assumptions of statistical independence across dimensions, no duplicate values over each dimension, and dimension domains totally ordered. Experiments verify that our approaches can estimate the space usage effectively over arbitrarily distributed data. To the best of our knowledge, this is the first work that attempts to address the issue and proposes effective approaches to solve it.

1. Introduction

Skyline queries [1] are very important for multicriteria decision-making applications, as the queries can return all the “best” elements which are not dominated by any other element. However, skyline queries may eliminate elements which are valuable but dominated by few other elements, for dimensions commonly can not cover all user’s consideration. Therefore, Papadias et al. [2] generalized the skyline to skyband, and a 𝑘-skyband query returns all the elements which are dominated by no more than 𝑘 of other elements.

By using the common hotel example in the literature, assuming that each hotel has the information of its distance from the beach and its price, and that one prefers the hotels which are cheap and close to the beach, Figure 1 demonstrates the difference between the skyline (the 0-skyband) and the 1-skyband. Three hotels, that is, 𝑎, 𝑑, and 𝑓, are returned by the skyline query, but additional four hotels, that is, 𝑏, 𝑐, 𝑒, and 𝑗, are returned by the 1-skyband query because they are dominated by only one of other elements. Buchta [3] proposed that the expected number of the skyline elements in a 𝑑-dimensional space which contains 𝑛 elements is Θ((ln𝑛)^𝑑−1/(𝑑−1)!); therefore, low-dimensional skyline queries commonly return a small number of skyline elements to the user, and some valuable elements may be eliminated, the reason is that each element has a high probability of being dominated by other elements in a low-dimensional space. Skyband queries may return the elements which are valuable but dominated by few other elements to the user, hence, are widely used by decision-making applications in low-dimensional spaces.

Figure 1 

Skyline versus 1-skyband.

Recently, the database research community witnessed a paradigm shift to continuous queries, and much attention has been put on sliding-window skyline queries [4, 5] in the stream environment. However, the issue of space usage estimation, which is very important for extending the query optimizer's cost model to accommodate skyline queries in the stream engine, is still left untouched. In this paper, we propose some effective approaches to estimate the space usage of sliding-window skyband queries. Since the skyline query is a special case of skyband queries, our proposed approaches can be naturally applied to sliding-window skyline queries as well.

Monitoring sliding-window skybands needs to extract all skyband elements from the live elements in the window and continuously report skyband changes as the window slides. In this paper, we first introduce the skyband sketch as the cost model and present effective policies for the sketch maintenance. As such, the skyband sketch has the quality of good space efficiency because it only stores the skyband elements along with the potential-skyband elements which do not belong to the skyband currently and are not guaranteed to be excluded from the skyband in their remaining lifespan. Next, under the assumption of statistical independence across dimensions, which is commonly used by query optimizers, and that no duplicate values exist over each dimension and domains are all totally ordered, we propose an approach for estimating the space usage of monitoring skybands over sliding windows. Experimental study verifies that our approaches can estimate the space usage effectively over arbitrarily distributed data. To the best of our knowledge, this is the first work that attempts to address the issue of space estimation and proposes effective approaches to solve it.

The rest of this paper is organized as follows. Section 2 summarizes the related work; Section 3 introduces some preliminary knowledge; Section 4 details our approaches for estimating the space usage; experimental results are given in Section 5 and followed by our conclusions in Section 6.

2. Related Work

Many algorithms have been proposed for computing static skylines, including the non-index-based algorithms [1, 6, 7] and the index-based algorithms [8–10], where the index-based algorithms uniformly outperform the non-index-based algorithms. Skyline computation under some certain conditions also received much attention, including skyline computation with partially ordered domains [11] and low-cardinality domains [12], subspace skyline computation [13, 14], skyline cube maintenance [15–19], and skyline computation in the distributed environment [14, 20–23]. Some skyline variations have also been proposed, including the 𝑘-dominant skyline [24], the top-𝑘 subspace skyline [25], the reverse skyline [26], the 𝑘 most representative skyline [27], the probabilistic skyline [28], and the skyband [2].

Under the assumptions of statistical independence across dimensions, no duplicate values over each dimension, and dimension domains being all totally ordered, the problem of estimating the number of the skyline elements, that is, the skyline cardinality, has been addressed in the works [3, 29, 30]. Chaudhuri et al. [31] relaxed the assumption of no duplicate values over each dimension by allowing two possible values (e.g., 0 and 1).

As stated before, continuous skyline queries over sliding windows in data streams [4, 5] have important applications such as environment monitoring and trends sensing. To accommodate skyline operator in the stream processing engine, the issue of space usage estimation needs to be solved. Motivated by this ambition, under the similar assumptions, we propose robust approaches to estimate the number of the skyband and potential-skyband elements over continuously distributed data.

3. Preliminaries

In this section, we present some preliminary results that will be used in the next section. In addition, we also describe a data structure called the skyband sketch. Theorem 3.1 characterizes the number of the elements in a finite set which just satisfy 𝑘 of the 𝑚 properties. It is based on the generalized form of the Inclusion-Exclusion Principle [32]. Similarly, Theorem 3.2 characterizes the number of the elements in a finite set which satisfy no more than 𝑘 of the 𝑚 properties; the theorem will be used for our theoretical analysis of the space usage in the next section.

Theorem 3.1. Suppose that 𝒮 is a finite set, 𝒫₁,𝒫₂,…,𝒫_𝑚 are 𝑚 properties, and 𝒮₁,𝒮₂,…,𝒮_𝑚 are 𝑚 subsets of 𝒮, where 𝒮_𝑖 consists of all those elements in 𝒮 with property 𝒫_𝑖. Let Δ(𝑚,𝑘) be the number of the elements in 𝒮 which just satisfy 𝑘 of the 𝑚 properties, it can be characterized as Δ(𝑚,𝑘)=^𝑚_𝑖=𝑘(−1)^𝑖−𝑘𝑖𝑘𝒯(𝑖), (3.1) where 𝒯(𝑖)(0≤𝑖≤𝑚) is characterized as follows: 𝒯(0)=|𝒮|,𝒯(1)=^𝑚_𝑖=1|𝒮_𝑖|,𝒯(2)=_{1≤𝑖1<𝑖2≤𝑚}|𝒮_𝑖1∩𝒮_𝑖2|,𝒯(3)=_{1≤𝑖1<𝑖2<𝑖3≤𝑚}|𝒮_𝑖1∩𝒮_𝑖2∩𝒮_𝑖3⋮|,𝒯(𝑚)=|𝒮₁∩𝒮₂∩⋯∩𝒮_𝑚|. (3.2)

Theorem 3.2. Suppose that 𝒮 is a finite set, 𝒫₁,𝒫₂,…,𝒫_𝑚 are 𝑚 properties, and 𝒮₁,𝒮₂,…,𝒮_𝑚 are 𝑚 subsets of 𝒮, where 𝒮_𝑖 consists of all the elements in 𝒮 which satisfy 𝒫_𝑖; the number of the elements in 𝒮 which satisfy no more than 𝑘 of the 𝑚 properties, that is, Γ(𝑚,𝑘), can be characterized as Γ(𝑚,𝑘)=𝒯(0)+^𝑚_𝑖=𝑘+1(−1)^𝑖−𝑘𝑘𝑖−1𝒯(𝑖), (3.3) where 𝒯(𝑖)(0≤𝑖≤𝑚) is the same as that in Theorem 3.1.

Proof. By Theorem 3.1, Γ(𝑚,𝑘) can be characterized as Γ(𝑚,𝑘)=^𝑘_𝑗=0Δ(𝑚,𝑗)=^𝑘𝑚_{𝑗=0𝑖=𝑗}(−1)^𝑖−𝑗𝑖𝑗=𝒯(𝑖)_{𝑘𝑖𝑖=0𝑗=0}(−1)^𝑖−𝑗𝑖𝑗𝒯(𝑖)+^𝑚𝑘_{𝑖=𝑘+1𝑗=0}(−1)^𝑖−𝑗𝑖𝑗𝒯(𝑖)=𝒯(0)+^𝑚_𝑖=𝑘+1(−1)^𝑖−𝑘𝑘𝑖−1𝒯(𝑖). (3.4) We have thus proved the theorem.

In a 𝑑-dimensional space, for simplicity and without loss of generality, an element 𝜉_• is said to dominate another element 𝜉_∘ if it is smaller than or equal to 𝜉_∘ over each dimension and strictly smaller than 𝜉_∘ over at least one dimension and is noted as 𝜉_•≻𝜉_∘. In a sliding-window, if no more than 𝑘 of other live elements can dominate an element, the element is a 𝑘-skyband element; if an element is not a 𝑘-skyband element and no more than 𝑘 of the succeeding elements can dominate it, the element is a potential-𝑘-skyband element.

Now we are able to describe a data structure called the skyband sketch for keeping the 𝑘-skyband elements or the potential-𝑘-skyband elements. The skyband sketch is a memory resident synopsis. The potential-skyband elements are the elements which do not belong to the skyband currently but are not guaranteed to be excluded from the skyband in their remaining lifespan. Hence the skyband sketch has the quality of good space efficiency for monitoring skybands over sliding-windows. The space usage in this paper is measured by the numbers of the skyband and the potential-skyband elements stored by the sketch.

Figure 2 shows the architecture of the skyband sketch; the sketch changes occur only when a new element arrives or a current skyband element expires. When a new element arrives, if no more than 𝑘 skyband elements can dominate it, it is probably a skyband element; otherwise, it is a potential-skyband element. If the new element appears to be a skyband element, all the skyband elements which are dominated by more than 𝑘 succeeding skyband elements and all the potential-skyband elements which are dominated by more than 𝑘 succeeding skyband and potential-skyband elements should be deleted because they will be dominated by the succeeding 𝑘 elements during their remaining lifespan; in addition, the skyband elements which are dominated by no more than 𝑘 succeeding skyband elements but are dominated by more than 𝑘 live skyband elements will appear to be potential-skyband elements. If the new element appears to be a potential-skyband element, all potential-skyband elements which are dominated by more than 𝑘 succeeding skyband and potential-skyband elements should be deleted. When a skyband element expires, all the potential-skyband elements which are dominated by no more than 𝑘 skyband and potential-skybad elements will appear to be skyband elements. In this paper, since we focus on the problem of space usage estimation, we leave out the detailed implementation issues of the skyband algorithm.

Figure 2 

The architecture of the skyband sketch.

4. Space Usage Estimation

In this section, we present our robust approaches for estimating the space usage of sliding-window skybands under the assumption of statistical independence across dimensions based on the preliminary results in the previous section.

4.1. Distribution-Constrained Data

Here, we give our theoretical analysis for the space usage of sliding-window skybands over data which is distribution constrained, that is, there are no duplicate values over each dimension. By mapping the problem of evaluating the number of the elements in a finite set which satisfy no more than 𝑘 of the 𝑚 properties to the problem of evaluating the probability that no more than 𝑘 of other 𝑚 elements can dominate an element, Lemma 4.1 gives the probability that at most 𝑘 of other 𝑚 elements in a 𝑑-dimensional space can dominate an element. Based on Lemma 4.1, Theorem 4.2 gives the expected number of the 𝑘-skyband elements in a sliding window which contains 𝑛   𝑑-dimensional live elements.

Lemma 4.1. Suppose that 𝜉₀,𝜉₁,𝜉₂,…,𝜉_𝑚 are 𝑚+1 elements in a 𝑑-dimensional space, under assumptions of statistical independence across dimensions, no duplicate values over each dimension, and data domains being all totally ordered; let 𝐷_𝑘(𝑚,𝑑) be the fact that no more than 𝑘 of other 𝑚 elements can dominate 𝜉₀, then the probability of 𝐷_𝑘(𝑚,𝑑), that is,  ℙ{𝐷_𝑘(𝑚,𝑑)}, can be characterized as ℙ{𝐷_𝑘(𝑚,𝑑)}=1+^𝑚_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑚𝑖𝑖−1. (4.1)

Proof. We map 𝒮, 𝒫_𝑖, and 𝒮_𝑖 in Theorem 3.2 to the full probability space, 𝜉_𝑖≻𝜉₀, and ℙ{𝜉_𝑖≻𝜉₀}, respectively; 𝒯(𝑖) is mapped to 𝒯^∘(𝑖), which can be characterized as 𝒯^∘𝒯(0)=1,_∘(1)=^𝑚_𝑖=1ℙ{𝜉_𝑖≻𝜉₀𝒯},_∘(2)=_{1≤𝑖1<𝑖2≤𝑚}ℙ{𝜉_𝑖1≻𝜉₀∧𝜉_𝑖2≻𝜉₀𝒯},_∘(3)=_{1≤𝑖1<𝑖2<𝑖3≤𝑚}ℙ{𝜉_𝑖1≻𝜉₀∧𝜉_𝑖2≻𝜉₀∧𝜉_𝑖3≻𝜉₀⋮𝒯},_∘(𝑚)=ℙ{𝜉₁≻𝜉₀∧𝜉₂≻𝜉₀∧⋯∧𝜉_𝑚≻𝜉₀}. (4.2)
Under assumptions of statistical independence across dimensions, no duplicate values over each dimension, and domains being all totally-ordered, an element has a 1/(𝑖+1)^𝑑 probability of being dominated by all other 𝑖 elements; therefore, 𝒯^∘(𝑖) can be further characterized as 𝒯^∘1(𝑖)=(𝑖+1)^𝑑𝑚𝑖. (4.3) By Theorem 3.2, ℙ{𝐷_𝑘(𝑚,𝑑)} can be characterized as ℙ{𝐷_𝑘(𝑚,𝑑)}=𝒯^∘(0)+^𝑚_𝑖=𝑘+1(−1)^𝑖−𝑘𝑘𝒯𝑖−1_∘(𝑖)=1+^𝑚_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑚𝑖.𝑖−1 (4.4) We have thus proved the lemma.

Theorem 4.2. Suppose that there are 𝑛  𝑑-dimensional live elements in a sliding window, under assumptions of statistical independence across dimensions, no duplicate values over each dimension, and dimension domains being all totally-ordered, the expected number of the 𝑘-skyband elements, that is, Ψ_𝑘(𝑛,𝑑), can be directly characterized as Ψ_𝑘(𝑛,𝑑)=𝑛+^𝑛−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑−1𝑘𝑖−1𝑛𝑖+1 (4.5) and can be recursively characterized as Ψ_𝑘(𝑛,𝑑)=Ψ_𝑘Ψ(𝑛−1,𝑑)+^𝑘(𝑛,𝑑−1)𝑛 (4.6) with initial conditions Ψ_𝑘(𝑛,1)=𝑘+1 where 𝑛≥𝑘+1 and Ψ_𝑘(𝑘+1,𝑑)=𝑘+1 where d≥1.

Proof. By Lemma 4.1, Ψ_𝑘(𝑛,𝑑) can be characterized as Ψ_𝑘(𝑛,𝑑)=𝑛⋅ℙ{𝐷_𝑘(𝑛−1,𝑑)}=𝑛+𝑛⋅^𝑛−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑖𝑖−1𝑛−1=𝑛+^𝑛−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑−1𝑘𝑛.𝑖−1𝑖+1 (4.7)Ψ_𝑘(𝑛,𝑑) can further be recursively characterized as Ψ_𝑘(𝑛,𝑑)=𝑛+^𝑛−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑−1𝑘𝑖−1𝑛𝑖+1=𝑛+^𝑛−2_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑−1𝑘+𝑖−1𝑛−1𝑖+1_{𝑛−1𝑖=𝑘+1}(−1)^𝑖−𝑘(𝑖+1)^𝑑−1𝑘𝑖𝑖−1𝑛−1=(𝑛−1)+^𝑛−2_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑−1𝑘+𝑛𝑖−1𝑛−1𝑖+1𝑛+1𝑛⋅^𝑛−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑−2𝑘𝑖−1𝑛𝑖+1=Ψ_𝑘Ψ(𝑛−1,𝑑)+^𝑘(𝑛,𝑑−1)𝑛 (4.8) with initial conditions Ψ_𝑘Ψ(𝑛,1)=𝑘+1(𝑛≥𝑘+1),_𝑘(𝑘+1,𝑑)=𝑘+1(𝑑≥1). (4.9) We have thus proved the theorem.

Theorem 4.3 shows that there exists inherent correlation between the expected number of the skyband elements in case of monitoring a (𝑑+1)-dimensional 𝑘-skyband over a sliding window which contains 𝑛 elements Ψ_𝑘(𝑛,𝑑+1) and the expected number of the elements stored by the skyband sketch in case of monitoring a 𝑑-dimensional 𝑘-skyband over a sliding window which contains 𝑛 elements Φ_𝑘(𝑛,𝑑), that is, Ψ_𝑘(𝑛,𝑑+1)=Φ_𝑘(𝑛,𝑑). In addition, the expected number of the potential-skyband elements in case of monitoring a 𝑑-dimensional 𝑘-skyband over a sliding window which contains 𝑛 elements Ω_𝑘(𝑛,𝑑) equals Φ_𝑘(𝑛,𝑑)−Ψ_𝑘(𝑛,𝑑). Therefore, by a minor revision, Theorem 4.2 can also be used to characterize the expected number of the potential-skyband elements.

Theorem 4.3. Under assumptions of statistical independence across dimensions, no duplicate values over each dimension, and domains being all totally-ordered, the expected number of the skyband elements in case of monitoring a (𝑑+1)-dimensional 𝑘-skyband over a sliding window which contains 𝑛 live elements, that is, Ψ_𝑘(𝑛,𝑑+1), equals the expected number of the elements stored by the skyband sketch in case of monitoring a 𝑑-dimensional 𝑘-skyband over a sliding window which contains 𝑛 live elements, that is, Φ_𝑘(𝑛,𝑑).

Proof. By Lemma 4.1, Φ_𝑘(𝑛,𝑑) can be characterized as Φ_𝑘(𝑛,𝑑)=𝑘+1+^𝑛_𝑗=𝑘+2ℙ{𝐷_𝑘(𝑗−1,𝑑)}=𝑛+^𝑛_𝑗=𝑘+2^𝑗−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑖𝑖−1𝑗−1=𝑛+^𝑛−1𝑛_{𝑖=𝑘+1𝑗=𝑖+1}(−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑖𝑖−1𝑗−1=𝑛+^𝑛−1_𝑖=𝑘+1((−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑖−1^𝑛−1_𝑗=𝑖𝑗𝑖)=𝑛+^𝑛−1_𝑖=𝑘+1(−1)^𝑖−𝑘(𝑖+1)^𝑑𝑘𝑖−1𝑛𝑖+1=Ψ_𝑘(𝑛,𝑑+1). (4.10) To see why the theorem holds, suppose 𝜉₁,𝜉₂,…,𝜉_𝑛 are the 𝑛 live elements in the sliding window, which are ascendingly ordered by the element sequence number, and 𝜉_𝑖1,𝜉_𝑖2,…,𝜉_𝑖𝑚, where 1≤𝑖₁<𝑖₂<⋯<𝑖_𝑚≤𝑛, are the 𝑚 elements stored by the skyband sketch for monitoring a 𝑑-dimensional 𝑘-skyband over the sliding window. We map each of the live element 𝜉_𝑖=⟨𝑥_𝑖1,𝑥_𝑖2,…,𝑥_𝑖𝑑⟩ into a (𝑑+1)-dimensional elements 𝜉^∘_𝑖=⟨𝑥_𝑖1,𝑥_𝑖2,…,𝑥_𝑖𝑑,1/𝜂_𝑖⟩, where 𝜂_𝑖 is the sequence number of the element, then 𝜉^∘_𝑖1,𝜉^∘_𝑖2,…,𝜉^∘_𝑖𝑚 are just the 𝑘-skyband elements in the (𝑑+1)-dimensional space.

4.2. A Dynamic Programming Algorithm

In this subsection, based on the theoretical analysis proposed in the above subsection, we propose an efficient dynamic programming algorithm to estimate the space usage. Since there exist inherent correlations among the expected number of the skyband elements, the expected number of the potential-skyband elements, and the expected number of the elements stored by the skyband sketch, we only consider how to estimate the number of the skyband elements.

Estimating the number of the skyband elements using (4.5) is infeasible in most cases because combination numbers are used to characterize the expected number of the skyband elements; for example, the number of the different ways of selecting 50 elements from 100 different elements can not be stored by a 64-bit integer. Based on (4.6), we can design a recursive algorithm to estimate the number of the skyband elements, which will not encounter integer overflow. The recursive algorithm can be characterized by a binary tree with the depth of max(𝑛−𝑘,𝑑), where 𝑛, 𝑘, and 𝑑 are the same as those in Theorem 4.2. Therefore, estimating the number of the skyband elements using the recursive algorithm has the computational complexity of Θ(2^{max(𝑛−𝑘,𝑑)}), which is unacceptable in most cases. Actually, there exists a large amount of duplicate computations in the binary tree; therefore, if duplicate computations can be eliminated, the computational complexity can be reduced. Algorithm 1 is a nonrecursive algorithm for estimating the number of the skyband elements, which is based on (4.6), and all the duplicate computations are eliminated. The algorithm is a dynamic programming algorithm [33], because although the algorithm is based on a recurrence, it is non-recursive, and each step of the algorithm gives an exact answer for the corresponding subproblem.

Input:  $n$: the number of the elements    $d$: the number of the dimensions $\hspace{1em}\hspace{1em}\hspace{1em}k$: the $k$-skyband Output: the expected skyband cardinality begin  if $n\le k+1$  then return $n$;  if $\mathrm{\hspace{0.17em}\hspace{0.17em}}d=1$  then return $k+1$;  for  $i=1$  to $d$ do $\alpha [i]\leftarrow k+1;$ $\hspace{1em}\xi \leftarrow 0;$  for  $i=k+2$  to $n$  do $\hspace{1em}\hspace{1em}\hspace{1em}\xi \leftarrow (\xi +1)$  mod 2;    if $\xi =1$  then $\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\beta [1]\leftarrow k+1$;     for $j=2$  to $d$  do $\beta [j]\leftarrow \beta [j-1]/i+\alpha [j];$    else $\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\alpha [1]\leftarrow k+1;$     for $j=2$  to $d$  do $\alpha [j]\leftarrow \alpha [j-1]/i+\beta [j];$    end  end  if $\xi =0$  then return $\alpha [d]$  else return $\beta [d];$ end

Algorithm 1 

Expected skyband cardinality: ${\mathrm{\Psi }}_k(n,d).$

Algorithm 1 functions as follows. First, two vectors 𝛼 and 𝛽 with size 𝑑 are created, and the values of 𝛼[1⋯𝑑] are initialized to Ψ_𝑘(𝑘+1,1⋯𝑑), respectively. According to the initial conditions, we have Ψ_𝑘(𝑘+1,1⋯𝑑)=𝑘+1, hence all the values of 𝛼[1⋯𝑑] are initialized to 𝑘+1. Then, we evaluate Ψ_𝑘(𝑘+2,1⋯𝑑) and store the values to 𝛽[1⋯𝑑], respectively. According to the initial conditions, we have Ψ_𝑘(𝑘+2,1)=𝑘+1, hence 𝛽[1] is set to 𝑘+1. According to the recurrence, we have Ψ_𝑘(𝑘+2,2)=Ψ_𝑘(𝑘+2,1)/(𝑘+2)+Ψ_𝑘(𝑘+1,2), that is, 𝛽[2]=𝛽[1]/(𝑘+2)+𝛼[2], hence we can evaluate Ψ_𝑘(𝑘+2,2) and store the value to 𝛽[2]. By the same principle, we may evaluate Ψ_𝑘(𝑘+2,3⋯𝑑) sequentially and store the values to 𝛽[3…𝑑]. We may continue to evaluate Ψ_𝑘(𝑘+3,1⋯𝑑) using the values in 𝛽[1⋯𝑑] and store the values of Ψ_𝑘(𝑘+3,1⋯𝑑) to 𝛼[1…𝑑], respectively, until we evaluate Ψ_𝑘(𝑛,1⋯𝑑) and store the values to 𝛼[1⋯𝑑] or 𝛽[1⋯𝑑]. At last, the value of 𝛼[𝑑] or 𝛽[𝑑] is returned as the value of Ψ_𝑘(𝑛,𝑑). It is apparent that the algorithm is space and time efficient, because the space complexity and the time complexity are Θ(𝑑) and Θ((𝑛−𝑘)𝑑), respectively.

5. Experiments

In this section, we verify our theoretical results on space usage estimation of the k-skyband operator monitoring skybands over sliding windows in the stream environment by extensive experiments. The algorithms have been implemented by the C++ programming language and run on a 2.0 GHz Intel CPU with 2 GB of memory, and the data over each dimension is generated by the (GNU Scientific Library GSL: http://www.gnu.org/software/gsl). We test the space performance in a lower dimensional (4-dimensional) and a higher dimensional (8-dimensional) space, respectively. According to the probability theory, if the data over a dimension is continuously distributed, the probability that there are duplicate values over the dimension is zero. Therefore, for each space, we generate a dataset; the data over the first dimension is normally distributed with 𝜎=500, and the data over other dimensions is normally distributed with 𝜎=100. At the same time, the sliding-window size increases from 500 to 1000 stepped by 50; for each step, we compute the maximal, average, and minimal skyband sketch size, number of the skyband elements, and number of the potential-skyband elements during the moving of the sliding window over one million elements. Since there is no previous work that evaluates the space usage over continuous data, thus we compare our corresponding theoretical results with the experimental results.

Figures 3 and 4 show the comparisons between experimental results and the theoretical results for 4-dimension space and 8-dimension space. We can see that the experimental results are almost the same as we expected in the theories. What is more is that the maximal values are not twice as much as the minimal value and they are all close to the theoretical results. For the given parament 𝑟 (𝑟-skyband) and 𝑑, both of the actual space usage and the estimated space usage increase with the window size, as more objects need to be evaluated. At the same time, the skyband cardinality also increases when the value of parament 𝑟 increases. The comparison between 4-dimension space and 8-dimension space, as Figures 3(a) and 4(e) show, illustrates that the skyband sketch size in high-dimension space is much more than that in low-dimension space, when the window size and the parament 𝑟 are given. This is because less elements are likely to be dominated by other objects in high-dimension space compared with in low-dimension space. As there are sufficient skylines for users to make a decision in the higher-dimensional space, skybands query shows its efficiency in low-dimensional space.

(a) Theoretical results

(b) Experimental results

(c) Experimental results

(d) Experimental results

(e) Theoretical results

(f) Experimental results

(g) Experimental results

(h) Experimental results

Figure 3 

Space performance of monitoring skylines over sliding windows in the stream environment in a 4-dimensional space where the data over each dimension is continuously distributed.

(a) Theoretical results

(b) Experimental results

(c) Experimental results

(d) Experimental results

(e) Theoretical results

(f) Experimental results

(g) Experimental results

(h) Experimental results

Figure 4 

Space performance of monitoring skylines over sliding windows in the stream environment in a 8-dimensional space where the data over each dimension is continuously distributed.

6. Conclusions and Discussions

Skyband query is of great importance for multi-criteria decision-making applications. To support skyband query in the stream engine, the problem of effective space usage estimation must be solved, which is important for extending the query optimizers cost model. In this paper, under the assumption of statistically independent [34, 35] across dimensions, no duplicate values over each dimension, and dimension domains being all totally ordered, we propose effective methods to address this issue; since the skyline query is just a special case of skyband queries, it is obvious that our approaches apply to sliding-window skyline queries either. We also put forward a dynamic programming algorithm to estimate the space usage, which is space and time efficient. In addition, if only the distribution function is given, we can also use the similar approach to evaluate the skyband cardinality over a space, where there are duplicate values over some dimensions. Finally, we carried out extensive experiments which verified that our proposed approaches can estimate the space usage accurately, hence, can be used to extend the optimizer's cost model for incorporating the skyband operator.

Acknowledgments

This work is partially supported by China “863” Hi-tech Program (Grant no. 2007AA01Z153), Zhejiang Provincial NSF (Grant no. Y1090096), and the National Natural Science Foundation of China (NSFC) under Grant no. 60573125 and 60873264.