Abstract
Resource location in structured P2P system has a critical influence on the system performance. Existing analytical studies of Chord protocol have shown some potential improvements in performance. In this paper a splay tree-based new Chord structure called SChord is proposed to improve the efficiency of locating resources. We consider a novel implementation of the Chord finger table (routing table) based on the splay tree. This approach extends the Chord finger table with additional routing entries. Adaptive routing algorithm is proposed for implementation, and it can be shown that hop count is significantly minimized without introducing any other protocol overheads. We analyze the hop count of the adaptive routing algorithm, as compared to Chord variants, and demonstrate sharp upper and lower bounds for both worst-case and average case settings. In addition, we theoretically analyze the hop reducing in SChord and derive the fact that SChord can significantly reduce the routing hops as compared to Chord. Several simulations are presented to evaluate the performance of the algorithm and support our analytical findings. The simulation results show the efficiency of SChord.
1. Introduction
The emerging applications of peer-to-peer technologies are providing users with cheap and powerful facilities for communication. Due to their decentralized nature, the peer-to-peer applications are enjoying growing popularity. Their architectures allow for wide availability of network services. Peer-to-peer networks fall into two categories, unstructured and structured networks. Structured peer-to-peer networks are appealing because they can provide decentralization, self-organization, failure resilience, and good worst-case lookup performance for applications. However, they suffer from high latencies in average cases. Since overlay connections span wide-area networks, the overall lookup time is strongly dependent on the latencies between the intermediate nodes of the lookup. It is important to minimize the hop count of the overlay network.
Chord [1] is a popular topology for structured peer-to-peer networks, where nodes are arranged in a ring. Chord methodology partitions the ring by introducing forward links so that message routes can skip consecutive nodes. These forward links roughly skip nodes in power-of-two segments around the ring and thereby make routing easy-to-implement and demonstrably efficient. The path length of routes in a Chord network with nodes is bounded by in the worst-case and /2 on average. A thorough analysis of the Chord approach can be found in [2]. Although in general the Chord approach is both interesting and worth-while, the actual implementation of the lookup protocol has some performance and reliability drawbacks.
Several methods [3–5] have been proposed to improve Chord lookup by exploiting the topological information of the underlying physical networks. Proximity neighbor selection (PNS) is one of the representative techniques that has become widely used. This seeks to select, among the possible next hops, the one that is the closest in the physical network or the one that represents a good compromise between progress in the identifier space and proximity. PNS methodology can benefit the overall latencies of the system when peer communication latencies vary significantly, but it may result in a greater number of routes or longer latency.
As an alternative to minimizing lookup latency via PNS, paper [6] describes an optimal routing scheme based on bidirectional lookup. This scheme calculates the optimal route with the least hop count by solving a coding problem based on the signed digit representations of minimal hamming weight. Although this optimal scheme can achieve hop count on average, any intermediate node failure during a lookup terminates the lookup and yields the worst-case hop count . F-Chord [7] proposes an alternative approach to reducing the hop count by producing fingers based on the Fibonacci number system. Inspired by F-Chord, paper [8] generalizes the design of the Chord finger table for fewer routing hops. The works [7, 8] improve the diameter by paying off a corresponding increase in the degree.
In this paper we propose a new splay tree-[9] based solution to arrange and maintain the finger table called SChord. Briefly, the main contributions of our work are as follows.(i)A simple and complete splay tree-based SChord structure is provided in this paper. To the best of our knowledge, it is the first work to improve Chord by introducing the tree structure into the design of finger table.(ii)The lookup performance of splay finger table (SFT) is evaluated. Theoretical analysis demonstrates the search cost on SFT for keys under Zipf distribution approaches , which is a notable improvement as compared to for lookup in Chord.(iii)We evaluate the availability of server selection in SChord and propose the server selection algorithm (SSA). Analysis shows that SSA is efficient as well as reliable even if there are a lot of more fingers in SChord node than Chord node has.(iv)We analyze the expected number of hops for the lookup of a random key both in Chord and SChord. Both theoretical analysis and simulation results show that SChord can significantly reduce the routing hops.
The remainder of this paper is structured as follows. Section 2 summarizes related work. The detailed design of our splay finger table and how to construct and search for the splay finger table are presented in Section 3. Section 4 describes the method of utilizing server selection via splay finger table. The results of experiments are presented in Section 5. Finally we present conclusions and potential future work in Section 6.
2. Related Works
Extensive work has been proposed to date multiobjective optimization for Chord-like DHTs. These studies fall into four categories, namely, optimizing the finger table, proximity neighbor selection, optimizing the routing algorithm, and data replication. We summarize the representative work for each of these categories.
One line of research improves routing via optimizing the finger table. Literature [7] proposes a family of routing schemes based on the Fibonacci number system, allowing to improve the maximum/average number of hops for lookups and the routing table size per node. It is shown that in F-Chord the diameter (i.e., the number of hops) is 0:72021 log and the average path length is 0:39812 log . Item [8] generalizes this result, showing how to construct an improved finger table when the objective is to reduce the number of hops, possibly at the expense of an increased size of the finger table. Monte Carlo simulation results show that the new proposed finger table provides superior routing performance and exhibits reduced sensitivity to failures.
Proximity routing used by recent papers in the area [3–5, 10] is suitable for distributed systems where underlying network connections incur long latencies. CFS [5] utilizes an internet coordinating system for its participating nodes and uses server selection to avoid visiting nodes with potential long latencies. This improves the overall routing performance. LPRS-Chord [11] discusses a random sampling technique to improve the lookup performance. It redesigns the nodes’ communication message to glean lookup traversing latency. Recursive lookups server selection exhibits a better scaling behavior. LPRS-Chord is fast, incurs little network overhead, and requires relatively few modifications to the existing Chord. The author modifies the finger selection algorithm in order to achieve a more balanced distribution. The advantage of this proposal as compared to previous approaches is that it does not add any overhead to the basic Chord algorithm. The key concept in the above strategies is that carefully preconfigured routing message can benefit the later operation.
Paper [6] describes a pioneering optimal routing scheme based on the signed digit representations of minimal hamming weight. The optimal routing scheme makes use of both clockwise and counterclockwise fingers and works out the path with the least hop count. Analysis shows that the optimal routing scheme can reduce the average hop count to but does not help in the worst-case, that is, hops. In addition, [12] brought us -hop lookup by issuing parallel queries.
The Beehive system [13] introduces a proactive replication framework for DHTs. The goal of this framework is to provide DHTs with -hop lookup performance for Zipf-like or power law and query distributions. One highlight of this study is that it presented an approach to alleviating the DHT hot spot problem through replication. In addition, utilizing this framework is independent of the underlying DHTs and therefore does not alter the DHT lookup algorithm.
3. Splay Finger Table
3.1. Overview Splay Finger Table (SFT)
SChord uses splay finger table (SFT) for routing. SFT has four attributes:(1): the splay tree that provides insertion and lookup of the desired key;(2)Count: a counter that records of the number of nodes in ;(3): the node that has the minimum ID value in ;(4): the node that has the maximum ID value in .
Splay tree is a self-adjusting binary search tree. The hierarchy of the node is closely related to the access frequency which leads to efficient access of the frequently visited nodes. On an -nodes splay tree, all these standard search tree operations have an average time bound of over a worst-case sequence of operations.
SFT has two functions: routing and caching. For routing, lookup can be performed on the SFT to find the closest preceding node to the desired key. The lookup is then directed to and the lookup continues until the desired key is found. Similarly, the key is routed through a sequence of nodes towards the destination. As with caching, when a lookup is performed on node , firstly searches its splay tree in SFT for the desired key. If the key is found, then lookup is done; otherwise directs the lookup to some node in its splay tree. After the lookup on is finished, inserts the pair to its splay tree, where node is the successor of key in the SChord ring. Each node can set a quota for its splay tree known as CACHE_MAX. When a node’s SFT reaches its quota, instead of being inserted into the node’s splay tree, the new pair will be dropped. The value of CACHE_MAX will be discussed in Section 4.
3.2. Settingup Splay Finger Table
The construction of SFT is similar to the construction of finger table in the original Chord. Each node maintains a SFT which contains a splay tree. When a node is created, it sets up an initial splay tree for its SFT by initiating lookup RPCs to find successors, that is, successor . It then inserts them into SFT’s splay tree. The insertion operation is the standard splay tree insertion. After the insertion, the other three attributes in SFT change accordingly. Figure 1 depicts an example of a Chord ring with .
In the example above, we examine the construction of node 0’s SFT step by step. When node 0 is created, all its attributes in the SFT are set to NIL. Then node 0 tries to construct its SFT by issuing a series of lookup RPCs. The fingers are then inserted to the splay tree using the splay tree insertion operation. The whole construction process is shown in Table 1.
3.3. Lookup on Splay Finger Table
From the previous section we learnt that the splay tree in a node’s SFT covers keys only in the interval . Thus, the lookup of key on SFT differs, according to whether the lookup key is in or out of the interval .
In the first case as illustrated in Figure 2, when node tries to resolve a lookup of key that falls out of interval , the lookup process continues on node . This is because the key is on the arc ; as far as node knows, is the nearest node to key in identifier space; hence the node for the next hop is .
The terminology which is used for describing the splay tree was introduced by Sleator and Tarjan [9]. Here we use the example in the preceding section and demonstrate how SFT changes along with the lookup process for key 1 on node 0.
Table 1 gives the ultimate state of the splay tree () in node 0. So the lookup for key 1 has two steps; first go right then left. Such right-left access combination is called zag-zig. In the splay tree lookup operation, a zag-zig move consists of two binary tree rotations. First it rotates right at the right child of the root node, and then it rotates left at the root node, as illustrated in Figure 3.
It is useful to use sentinel node pointer to replace the NIL children pointers in the leaf nodes. So after the splay search process is done and no appropriate node has been found, then the sentinel node becomes the root of the splay tree. This sentinel root node is then removed by replacing it with the largest node in its left subtree, as shown in Figure 4. After the removal, the lookup will be directed to the new root node.
When the lookup is finished, several additional postoperations are needed to update the splay tree. Since the SFT caches historical search results, for a lookup of key which does not exist in the splay tree, the key and the lookup result pair are later inserted into the splay tree unless SFT reaches its quota, that is, CACHE_MAX. The insertion operation on splay tree consists of two steps: splay searches the key to be inserted and then replaces the root node with the pair node if key is not found. The following is the lookup algorithm on SFT described in C-like pseudocodes; we name it slookup (see Algorithm 1).
| ||||||||||||||||||||||||||||||||||||
3.4. Lookup Performance on Splay Finger Table
The finger table in Chord is stored in a form of sequence list or array which supports randomly access of elements. When a lookup is performed, Chord simply does a linear search on finger table by examining entry from the last to the first. Since Chord uses consistent hashing to generate nodes’ identifiers, we can infer its properties from the consistent hashing. For each node , the identifier difference between and its successor is roughly . So the number of distinct fingers in finger table is approximately . From the definition of finger table in Chord we may infer that the th finger in ’s finger table is responsible for up to keys. The average cost of the lookup for a random key on node can be calculated as follows:
Thus the cost of the lookup in Chord for keys under uniform distribution is . The cost of SFT lookup is composed of two parts: the splay tree operations, including insertion and retrieval of keys, and the update operations for SFT attributes. The SFT update operations contribute to the constant factors of the overall cost as illustrated in the pseudocode of slookup, so we only need to examine the cost of splay tree operations.
We use amortized algorithm analysis to determine the behavior of splaying over long sequences of operations. The amortized cost is defined to be for , where is the actual cost of operation and is the credit balance after the operation . So the total actual cost and total amortized cost of a sequence of operations on a data structure are related by
Let denote the number of nodes in the subtree rooted at node after step of the splaying process, then we define the rank at each step of to be
We assume that, after splaying steps, ends up as the root. Hence we obtain that the total amortized cost is
Thus the amortized cost of an insertion or retrieval with splaying in a binary search tree with nodes does not exceed upward moves of the target node in the tree.
The total complexity of a sequence of insertions or retrievals with splaying in a binary search tree that never has more than nodes does not exceed upward moves of a target node in the tree.
Here we recall the static optimality theorem in [9].
If every item is accessed at least once, then the total access time is
Consider that the nodes accessed are under Zipf distribution. We define the rank of node to be , according to Zipf’s law , where and are constants that characterize the distribution. So the amortized access time becomes where ; thus the average access time in the worse-case is
In the latter section we will discuss the upper bound for , that is, CACHE_MAX which is a constant. So the net potential drop over a long sequence of accesses is bound to .
In contrast, we determine the lower bound for the lookup process in Chord when keys are under Zipf distribution. Assume that there are resources with distinct ranks. Similarly, let be the rank of the th resource and let denote an arbitrary permutation of of integers in set , for . Then the expected cost of the Chord lookup for keys under Zipf distribution follows where and is the number of distinct entries in the node’s finger table. We have derived the value of ; that is, ; is the number of nodes in the Chord ring. Since the resources are finite and is a constant, the lower bound is then .
Based on the above analysis, when the lookup key is under uniform distribution, the expected cost of a search operation on splay finger table is compared to for the search in Chord. However, in the real world, the lookup key is nonuniform, approximately under Zipf distribution. In this case the average cost of slookup approaches , which is a notable improvement as compared to for lookup in Chord. This is also confirmed in our experiment.
4. Server Selection
The use of server selection in the lookup layer strives to reduce the overall latency by choosing the node that has the least estimated lookup time (ELT) as the next hop node.
4.1. Background: Server Selection in Chord
When a lookup is performed, the original node tries to find the predecessor of the lookup key by issuing a RPC to a node , and then returns the closest preceding nodes in its finger table to . This process is reiterated until the desired successor is found. Chord adopts this lookup method to minimize the number of necessary hops along the lookup path. Therefore the overall lookup time strongly depends on the intermediate nodes among the lookup path.
Potentially the server selection could be used at each step of the lookup process to reduce the overall lookup latency. Namely, when an intermediate node tries to find the node for the next hop, instead of picking the node with the closest ID in its finger table, it estimates the time left to finish the lookup process for each finger that precedes the lookup key. So the next hop node should be the one has the least ELT.
Figure 5 illustrates a situation of the choice of a potentially better lookup path. When the client (node ) parses the lookup of target (node ), it compares to its fingers and finds two matching nodes , . The direct one-trip time (OTT) between and could be estimated via network coordinates or querying ’s historical latency database. So the ELT for the rest of the lookup could be calculated as
It is better to pick node rather than if .
We denote by the overall latency of the lookup for id that starts from node . Then is estimated by
Having be the OTT between and and be the average per hop latency which might be obtained from the node’s historical RPC latency data, is the estimated number of hops left from to the lookup finishes. For an -node Chord ring can be calculated as where “” is the binary right shift operation and ones() function counts the number of significant bits of integer in binary.
To utilize server selection on the original Chord, at each step of the lookup process, the node for this hop examines the fingers preceding the desired key in its finger table and chooses the one that has the least ELT as the next hop node. However, for server selection on SFT, this is a different scenario.
4.2. Server Selection via Splay Finger Table
Unlike the finger table in Chord, which has fixed size in fingers, the number of fingers in SFT is dynamic, varying from to CACHE_MAX. So the number of predecessors of the desired key at each step of the lookup could be large. A new method needs to be discovered so that the server selection can be performed as efficiently as Chord’s.
The algorithm we proposed here is inspired by probabilistic analysis. Consider the situation that at some step of a lookup on node; examines the predecessors in its SFT of the desired key and assumes that there are of such predecessors. So will then pick one of these predecessors that have a smaller ELT as the next hop node. By saying “smaller ELT,” we mean the node which has an ELT that is smaller than ’s ELT, where is the node in SFT whose ID is the closest to the desired key. Let be the fingers in SFT that precedes the desired key; the following gives the server selection algorithm (SSA) on SFT (see Algorithm 2).
| ||||||||||||||||||
The idea we used in SSA is to cancel the first candidates, namely, , and then pick the first candidate in such that is less than all . If no satisfies, then SSA will pick by default.
We shall analyze each possible value of and the probability that SSA picks the node that has the least ELT. In this case we say that SSA made the best choice. Then we will choose the best possible and implement the server selection with that value. Assume that is fixed for the moment. Let denote the minimum cost among the candidates 1 through . Let be the event in which SSA succeeds in making the best choice and let be the event in which SSA succeeds when is the one that has the least ELT. Since the various are disjoint, we have . SSA never succeeds if the best one is in ; we have , for . Thus we obtain
In order for to succeed, candidate should have the least ELT; in addition, none of the candidates through have been chosen. Since is under random distribution and OTTs are measured with high precision, we can tell that there are no such two candidates and such that . Hence we have
Since the following inequalities hold for
The lower and upper bounds for are then obtained as follows:
By differentiating the lower bound expression with respect to , we obtain
Setting this derivative equal to 0, we see that the lower bound for is maximized when . Thus the SSA will succeed in making the best choice with probability at least .
Now we determine the cost of SSA in iterations. The expected number of iterations in SSA with respect to is
Substitute with ; the expression of becomes When grows large enough, is approximately
Let = . is the sever selection cost of SSA in SChord. Then . Therefore SSA is faster than server selection in Chord which calculates for every candidate before making a choice, resulting in iterations. Furthermore, by setting CACHE_MAX to (or ), where is the bit length of the node’s identifier, SSA could perform as efficient as the server selection in Chord. Better still, SSA can find the candidate that has the least ELT with probability at least .
4.3. Hop Reducing
By extending the finger table with historical lookup pairs, at each step of the lookup, the node that resolves the lookup could potentially take larger advances towards the desired key. For example, when SFT in a node is initially built, it has exactly the same fingers as the finger table does in Chord. SFT then grows with the number of lookups done on node since caches the historical lookup results with the corresponding keys (see Section 3). Thus for some lookup of key , if falls between and , where , with some probability, there exists a cached key in ’s SFT such that , so will choose the node in its SFT whose ID is as next hop node, instead of in Chord.
We now use (14) to determine the expected number of hops for the lookup of a random key both in Chord and SChord, respectively. We denote by the number of hops for a lookup in Chord and in SChord. The ones() function is provided as in Algorithm 3.
| ||||||||||||||
The cost of ones() function is determined by the number of significant bits in . Assume that there are of them; then the cost of ones() is . Thus, for , , the expected cost of ones() is .
Lemma 1.
Consider
For any node , let be the event in which is responsible for the desired key. In order for to succeed, key should be in ; hence the probability in which happens is
and the expectation for follows
Applying Lemma 1 we obtain
Lemma 2. For a SChord node with fingers, the cost of a lookup for any key never exceeds .
Proof. SChord contains all the fingers borrowed from Chord and cached fingers. From the property of consistent hashing, we know that the cached fingers divide the ID space to segments with each segment length in bits. After the first lookup step is done, the difference between current hop and the desired key in ID space is then less than a segment. Finally, the total number of hops, including the initially first lookup step, never exceeds .
Let us examine the expression of . By substituting in the expression with , when and , according to Lemma 2, we obtain an upper bound for
So the expected number of hops reduced by slookup as compared to lookup is
Let denote the percentage of the promotion; then we have and recall the fact that , so , so we obtain a lower bound for :
Since the latency for each routing hop is relatively fixed, we may infer that when , there are up to 1384 nodes, which is also the approximate number of peers in a popular Bit-Torrent swarm. SChord increases the lookup performance by 62% with respect to Chord. For the network of nodes, SChord increases the lookup performance by 26% as compared to Chord.
5. Experiments
In this section, we conduct experiments with the SChord by simulation. The protocol is implemented as recursive style so each intermediate node forwards a request to the next node until it reaches the successor of the desired key, and server selection is optional at each of these intermediate nodes.
5.1. Hop Estimation
We first consider the ability of (14) to predict the lookup hops accurately. Equation (14) is used both in the server selection and in the analysis of hop reducing. So the accuracy of (14) is crucial. We test (14) in networks consisting of nodes varying from 25 to 1000. For each network size, we perform 1000 lookups of random keys. The error of the 1000 lookups is calculated as follows.
Let be the actual number of hops for the th lookup and let be the predicted number of hops for this lookup, for . We have
Figure 6 plots the error of networks with size varying from 25 to 1000 nodes. Thus for the majority of networks the lookup errors are below one hop and for all of the networks the errors are below 1.2. We draw the conclusion that (14) is relatively accurate.
5.2. SFT Lookup Performance
In this experiment, we evaluate the finger table lookup performance in Chord and SFT lookup performance in SChord, respectively. In the real world, the lookup keys are under Zipf distribution. So we randomly generate a total of 125 resources ranking from 1 to 125; then the lookup keys are restrained to those 125 resources. In this case, we test the lookup time for lookups on finger table and SFT over up to 1,000,000 lookups.
Figure 7 plots the lookup time curves. We can see that the SChord lookup curve is relatively steady and the Chord lookup curve grows faster. Therefore, SChord can achieve much better lookup performance than Chord on a large scale of lookups.
5.3. Hop reducing
We shall verify the feasibility to predict the expected number of hops involved in one single lookup or slookup. According to the previous induction we know that the expected number of hops involved in one single lookup should be . This is based on the hypothesis that (14) holds for arbitrary IDs. Besides, the previous induction gives us an upper bound for the expected number of hops involved in one single slookup; that is, . Finally and are the predicted hops; we shall match them with the real hop data.
Figure 8 compares the experiment hop data for network size 1 through 10,000 with the values evaluated by the expressions of and . For each network size, we randomly choose 10,000 nodes with each node resolving a lookup for a single random key. The figure shows that the upper bound works well when there are more than 1000 nodes in the SChord ring. In addition, the difference between estimated lookup hops and expected lookup hops is less than one hop; thus the prediction is quite accurate.
Since slookup caches the historical data, the number of fingers in a node’s SFT increases with the lookups performed on this node until count reaches CACHE_MAX. Meanwhile, the expected number of hops for one lookup on this node decreases while Chord has no way of reducing hops so the expected cost of lookup is only relevant to the network scale. Figure 9 is the comparison of the average number of hops required between lookup and slookup in two different networks of 1389 and 11072 nodes, respectively. In this experiment, for each network scale, we randomly choose a node to perform 1 through 10,000 lookups and slookups and then record the average number of hops for lookup and slookup.
Accordingly, Figure 10 plots the number of hops reduced by slookup compared to lookup in Figure 9. The preceding induction gives a lower bound for it. From the figure we learn that this lower bound works well after some initial slookup done on the node. In this test, the lower bound holds for around 4000 initial lookups. For systems where data are accessed in blocks or lookup operations are resolved frequently, this initial lookup requirement could be easily met.
In conclusion, the experiment results confirm that the lower bound for reducing hops is possible in practice. Hence, SChord increases the lookup operation performance by 62% at the maximum when there are 4c nodes in the network. Moreover, when the network size grows to as many as 16,000,000 nodes, SChord could still increase the lookup operation performance by 26% which is an immense improvement.
6. Conclusions
SChord is a highly scalable, available, and efficient resource location protocol. It takes advantage of splay tree where lookup caching can be used to accelerate later lookups. So the lookup performance increases based on the total number of lookups done on the node. Theoretical analysis and simulation results both confirm the fact that the lookup performance in SChord has increased up to 62% in comparison with Chord. Moreover with larger CACHE_MAX the lookup performance can be further enhanced in SChord. Detailed studies in the design space of SChord also bring interesting and useful results in building a load balance P2P system in practice.
Further study will focus on recycling mechanism for SFT so that SChord can adjust to resource change in the dynamic network circumstance.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This research is supported by the National Natural Science Foundation of China, under Grant no. 61363021.