Topological Properties of Hierarchical Interconnection Networks: A Review and Comparison

Abstract

Hierarchical interconnection networks (HINs) provide a framework for designing networks with reduced link cost by taking advantage of the locality of communication that exists in parallel applications. HINs employ multiple levels. Lower-level networks provide local communication while higher-level networks facilitate remote communication. HINs provide fault tolerance in the presence of some faulty nodes and/or links. Existing HINs can be broadly classified into two classes. those that use nodes and/or links replication and those that use standby interface nodes. The first class includes Hierarchical Cubic Networks, Hierarchical Completely Connected Networks, and Triple-based Hierarchical Interconnection Networks. The second HINs class includes Modular Fault-Tolerant Hypercube Networks and Hierarchical Fault-Tolerant Interconnection Network. This paper presents a review and comparison of the topological properties of both classes of HINs. The topological properties considered are network degree, diameter, cost and packing density. The outcome of this study show among all HINs two networks that is, the Root-Folded Heawood (RFH) and the Flooded Heawood (FloH), belonging to the first HIN class provide the best network cost, defined as the product of network diameter and degree. The study also shows that HFCube(𝑛,𝑛) provide the best packing density, that is, the smallest chip area required for VLSI implementation.

1. Introduction

In nonshared memory multiprocessors systems, interconnecting processors via message passing can be either directly, which is costly for large number of processors, that is, 𝑂(𝑁²), where 𝑁 is the total number of processors, or indirectly by routing messages via intermediate processors. Three important factors should be considered at the design stage of indirect connection: minimizing the message delay, reducing the cost, and maximizing the reliability. It has been shown that hypercube networks (HCNs) are highly efficient in connecting multicomputer networks [1]. However, HCNs have the limitation that increasing the number of nodes requires a change in the basic node configuration and causes an exponential increase in the number of links. This limits the applicability of the hypercube to very large systems. Also, there is some locality of communication among nodes of the hypercube which is not exploited for performance gains. An interesting recent result in the context of fault-tolerant cycle embedding in hypercube is that of recursive embedding of a longest cycle into 𝑛-dimensional hypercube in a way to tolerate at most (2𝑛−4) faulty nodes [2]. This is to be compared to its predecessor result of tolerating a maximum of (𝑛−1) faulty nodes.

Hierarchical interconnection networks (HINs) provide an opportunity for taking advantage of such factors [3]. These networks employ multiple levels in which lower-level networks are used to provide local communication and higher-level networks are used to facilitate remote communication. HINs provide fault tolerance through duplicating the higher-level network, known as the replication technique (see Figure 1), or through the use of standby spare interface nodes (see Figure 2).

Figure 1 

The replication technique.

Figure 2 

The standby spare technique.

In recent years, hierarchical interconnection networks have attracted increasing attention. This is because they provide a framework to design networks with reduced link cost. They also take advantage of the inherent synergy among communicating tasks in parallel applications. At the lowest level, HINs provide clusters of individual nodes, with nodes in each cluster connected by a network. This network is called a level-one network. At the next level, groups of clusters are connected by a level-two network. The topology at each level can be the same or can be different. If the topologies at all levels are the same, then the network is called homogenous hierarchical interconnection network; otherwise, the network is called a heterogeneous hierarchical interconnection network [3].

Hierarchical interconnection network was originally introduced by Dandamudi and Eager in [3] using only two levels. Wei and Levy in [4] proposed a class of general hierarchical interconnection networks in which more than one node from each cluster can act as interface nodes. The procedure for construction the HINs in this case is as follows. The total number of nodes 𝑁 is divided into 𝐾₁ clusters of 𝑁/𝐾₁ nodes each. Each cluster of 𝑁/𝐾₁ nodes is connected to form a level-1 network. The nodes in every cluster are ordered in the same way. Then 𝐼₁ nodes, 1≤𝐼₁≤𝑁/𝐾₁, from each cluster are selected to act as interface nodes. To construct level-2 networks, these interface nodes are first divided into 𝐼₁ groups. Each group consists of 𝐾₁ nodes, which are from 𝐾₁ different clusters. Then, the 𝐾₁ nodes of each group are again divided into 𝐾₂ clusters of 𝐾₁/𝐾₂ nodes each. Each cluster is connected to form a level-2 network, and 𝐼₂ nodes, 1≤𝐼₂≤𝐾₁/𝐾₂, are selected as the interface nodes to construct the level-3 networks and so on. The interconnection networks used to construct clusters at different levels may have the same or different topologies (see Figure 3).

Figure 3 

A HIN based on Hypercubes [4].

A number of hierarchical interconnection networks have been proposed in the literature [5–13, 15–17]. These can be classified into Class 1, such as Block-Shift Network (BSN), Extended Hypercube (EH), deBruijn with Hypercube (dBCube), Hierarchical Hypercube (HHC), Hierarchical Cubic Network (HCN), Hierarchical Completely Connected Networks (HCC), HIN with Folded Hypercubes (HFCube), Folded Heawood (FolH), Root-Folded Heawood (RFH), Recursively Expanded Heawood (REH), Flooded Heawood (FloH), Triple-based Hierarchical Interconnection Networks (THIN), and Rectangular Twisted Torus Meshes (RTTM); Class 2, such as Modular Fault-Tolerant Hypercube Networks (MFTHN) and Hierarchical Fault-Tolerant Interconnection Network (HFTIN). This paper presents a study and a comparison of the topological properties of these hierarchical interconnection networks. The topological properties included in this study are defined below.

Definition 1. Network degree is defined as  the maximum number of ports per node for all nodes in the network.

Definition 2. Network diameter is defined as  the maximum shortest path consisting of distinct hops, between any two nodes in the network.

Definition 3. Network cost is defined as the product of network diameter and degree.

Definition 4. Packing density is defined as the ratio of the number of nodes of a network to its cost.

This paper is organized as follows: a description of HINs based on the replication technique is introduced in Section 2. Section 3 provides an introduction to HINs based on the standby spare interface node technique. Section 4 provides the detailed comparison conducted among HINs. Finally, Section 5 concludes the work.

2. HINs Based on the Replication Technique

The replication technique aims at avoiding excessive traffic on intercluster links by duplicating the level-two network and having two or more interface nodes per cluster. A number of the proposed hierarchical interconnection networks in the literature are based on the replication technique [5–9, 12, 13, 15, 17]. The following subsections provide description of each category of these HINs.

2.1. Block Shift Network (BSN)

The Block-Shift Network (BSN) was introduced by Pan in 1991 [5]. Neighboring nodes in a BSN are tightly coupled while remote nodes are loosely coupled. This property makes the BSN topology suitable for localized traffic patterns, a property observed in a number of applications. There are 𝑁=2^𝑛 nodes in a BSN. In each step of constructing a BSN network, only 𝑎 bits can be changed within the section of the right most 𝑏 bits. This gives the resulting network the label BSN(𝑎,𝑏). Changing the two parameters 𝑎 and 𝑏 defines the network connection type (see Figures 4(a) and 4(b)). The degree of a BSN(𝑎,𝑏) is computed as (2^𝑎−1)∗(𝑏/𝑎)+2 and its diameter is (1+(𝑏/𝑎))∗⌈𝑛/𝑏⌉. The design of the BSN is greatly motivated by its flexibility. This is because the parameters 𝑎 and 𝑏 can be changed in accordance with the performance and cost requirements of a given network. In addition, a BSN is scalable in the sense that changing the size of the network does not require changing the hardware of its nodes. A number of existing networks can be considered as special cases of the BSN. For example, BSN(1,1) is the shuffle-exchange network, BSN(1,𝑛) is the 𝑛-dimensional hypercube, while BSN(𝑛,𝑛) is the complete network.

(a)

(b)

Figure 4 

(a) A BSN(1,2) with 𝑁=16, (b) A BSN(2,2) with 𝑁=16 [5].

2.2. Extended Hypercubes (EH)

This is a hierarchical and recursive structure with a constant predefined building block. The EH was introduced by Kumar and Patnaik in 1992 [6]. The basic module EH(𝑘,1) of an EH consists of 𝑘-cube of Processor Elements (PEs) and a Network Controller (NC). The NC is used as a communication processor to handle intermodule communication. The basic EH module is said to be of degree one. The EH(𝑘,𝑙) architecture can be constructed by connecting the basic modules up to 𝑙 levels as shown in Figure 5. In this figure, two levels (𝑙=2) of the basic EH(3,1) are connected to form an EH(3,2). The degree of an EH(𝑘,𝑙) is 𝑘+1 while its diameter is 𝑘+2(𝑙−1).

Figure 5 

An EH(3,2) constructed from EH(3,1) [6].

2.3. dBCube

The dBCube is a compound graph consisting of a deBruijn graph in which each node is replaced by a hypercube cluster [7]. A dBCube(𝑐,𝑑) is defined as having 𝑐 cubes/cluster (or the size of the deBruijn graph) and that each cube is of dimension 𝑑 (see Figure 6). In Figure 6, each node is a 2-cube (𝑑=2) and there are eight cubes/cluster (𝑐=8). The degree of a dBCube(𝑐,𝑑) is 𝑑+1 and its diameter is (1+log(𝑐)).

Figure 6 

A dBCube(8,2) [7].

2.4. The Hierarchical Hypercubes

Malluhi and Bayoumi introduced the Hierarchical Hypercube Network of order 𝑛 (𝑛-HHC) [8]. The structure of an 𝑛-HHC consists of three levels of hierarchy. To simplify the description of HHC structure, assume that the order 𝑛=2^𝑚+𝑚 for a nonnegative integer 𝑚. At the lowest level of hierarchy, there is a pool of 2^𝑛 nodes. These nodes are grouped into clusters of 2^𝑚 nodes each, and the nodes in each cluster are interconnected to form an 𝑚-cube called the Son-cube or the 𝑆-cube. The set of the 𝑆-cubes constitutes the second level of hierarchy. A father-cube or the 𝐹-cube connects the 2^𝑛−𝑚=2^2𝑚𝑆-cubes in a hypercube fashion. Figure 7 shows a 5-HHC. The degree of an HHC is 𝑚+1 while its diameter is 2^𝑚+1.

Figure 7 

A 5-HHC [8].

2.5. Hierarchical Cubic Networks (HCN)

The Hierarchical Cubic Network HCN(𝑛,𝑛) is a hierarchical network consisting of 2^𝑛 clusters, each of which is an 𝑛-dimensional hypercube [9]. Each node in the HCN(𝑛,𝑛) has (𝑛+1) links connected to it. Figure 8 shows HCN(2,2). The HCN uses almost half as many links as a comparable hypercube and yet has a smaller diameter than a comparable hypercube while emulating desirable properties of a hypercube. The degree of an HCN(𝑛,𝑛) is 𝑛+1 while its diameter is 𝑛+⌊(𝑛+1)/3⌋+1. In [18], a maximal number of node-disjoint paths have been constructed between each pair of distinct nodes of the HCN(𝑛). The maximum length of these node-disjoint paths (𝑛-fault diameter) is bounded above by 𝑛+⌊𝑛/3⌋+4. The (𝑛+1)-wide diameter of the HCN(𝑛) is shown to be ⌊𝑛/3⌋+3. These results represent about two-thirds those of a comparable hypercube.

Figure 8 

A HCN(2,2) network [9].

In [19], fault-free Hamiltonian cycles in an HCN(𝑛) with 𝑛−1 link faults have been constructed using Gray codes. Since the HCN(𝑛) is regular of degree 𝑛+1, the result shown in [19] is optimal. The longest fault-free cycles of length 2^2𝑛−1 in an HCN(𝑛) with a one-node fault and fault-free cycles of length at least 2^2𝑛−2𝑓 in an HCN(𝑛) with 𝑓-node faults have been also constructed in [19], where 2^2𝑛 is the number of nodes in the HCN(𝑛), 𝑓≤𝑛−1 if 𝑛=3 or 4 and 𝑓≤𝑛 if 𝑛≥5.

2.6. Generalized Hierarchical Completely Connected Networks (HCC)

Takabatake et al. proposed the generalized hierarchical completely connected networks (HCCs) in [10]. The construction of an HCC starts from a basic block (a level-1 block) which consists of 𝑛 nodes of constant degree. Then a level-ℎ block for ℎ≥2 is constructed recursively by interconnecting any pair of macro nodes (𝑛 level-(ℎ−1) blocks) completely. An HCC has a constant node degree, which is 𝑟+1, regardless of the size of the network. A generalized HCC_G and the concept of HCC are illustrated in Figure 9.

Figure 9 

An HCCG(3) [10].

2.7. HIN with Folded Hypercubes as Basic Clusters (HFCube)

A hierarchical interconnection network using folded hypercubes as basic clusters is proposed in [11] and is denoted as HFCube. An HFCube(𝑛,𝑛) has 2^𝑛 clusters, where each cluster is a folded hypercube FHC(𝑛). Each node in the HFCube(𝑛,𝑛) has 𝑛+2 links connected to it. Accordingly, the degree and diameter of a HFCube(𝑛,𝑛) is 𝑛+2 and 𝑛+1, respectively. An HFCube(3,3) is illustrated in Figure 10.

Figure 10 

An HFCube(3,3) network [11].

2.8. Heawood HINs

Jan et al. in [12] proposed four hierarchical interconnection networks based on Heawood graphs. These include (1) Folded Heawood (FolH), (2) Root-Folded Heawood (RFH), (3) Recursively Expanded Heawood (REH), and (4) Flooded Heawood (FloH). A Heawood graph has fourteen nodes with twenty-one links connecting them as shown in Figure 11. Each node in a Heawood network has three neighboring nodes. For any pair of nodes, there are three paths for routing a message between them and the minimum length of cycles containing any pair of nodes is 6. Folded Heawood HIN is based on the same concept used in the definition of folded Petersen network [11]. A two-dimensional folded Heawood network are shown in Figure 12 (note that the details of only two neighbors in Figure 12 is shown for the sake of brevity). Root-Folded Heawood HIN, alternatively, uses only a single link between neighboring nodes as shown in Figure 13. The recursive expansion method has been applied with the Recursively Expanded Heawood HIN to avoid bottlenecks caused by the root nodes of the Root-Folded Heawood HIN (see Figure 14). Similarly, the Flooded Heawood HIN is obtained by recursively expanding the basic Heawood network as illustrated in Figure 15.

Figure 11 

A Heawood graph [12].

Figure 12 

A 2-dimentinal FolH HIN [12].

Figure 13 

A 2-dimentinal RFH HIN [12].

Figure 14 

A 2-dimentinal REH HIN [12].

Figure 15 

A 3-dimentinal FloH HIN [12].

2.9. Triple-Based HIN (THIN)

The Triple-based HIN (THIN) was proposed by Qiao et al. in [15]. Figure 18 shows examples of THINs with different levels. In Figure 18, (a) shows a level 0 THIN, (b) shows a level 1 THIN, (c) shows a level 2 THIN, and (d) shows a level 3 THIN. The topology of THIN is very simple and the node degree is very low. THIN has obviously a hierarchal, symmetric, and scalable characteristic. The degree and diameter of THIN is 3 and 2^log𝑁3−1, respectively.

2.10. Rectangular Twisted Torus Meshes (RTTMs)

The Rectangular Twisted Torus Meshes (RTTMs) have been proposed in [16]. At the lowest level of RTTM network, the Level-1 subnetwork, also called a Basic Module, consists of a mesh connection of 2^𝑚×2^𝑚 nodes. Successively higher-level networks are built by recursively interconnecting 𝑎×2𝑎 next lower level subnetworks in the form of a Rectangular Twisted Torus, which consists of a rectangular array of 𝑎 rows and 2𝑎 columns (see Figure 19). An appealing property of the RTTM network is its smaller diameter and shorter average distance, which implies a reduction in communication delays. An (𝐿,𝑎,𝑚)-RTTM denotes an RTTM network with 𝐿 levels in its hierarchy. The number of nodes in RTTM is (𝑎×2𝑎)^(𝐿−1)×2^2𝑚. The degree and diameter of RTTM are 4 and (2𝐿−3)𝑎+2^𝑚+2−4, respectively.

3. HINs Based on the Standby Spare Interface Node Technique

Standby spare interface node technique provides a standby spare interface node to avoid the intercluster disconnection caused by interface node failure. Abdulla in [13] proposed the Modular Fault-Tolerant Hypercube Networks (MFTHN). MFTHN is a hierarchical network using the spare standby technique based on a basic block called Fault-Tolerant Basic Block (FTBB). The FTBB is a binary hypercube of dimension 𝑑 to which a spare node has been added to provide fault tolerance, which is connected to all the nodes of the hypercube. Large hypercubes can be built using FFTBs by utilizing the recursive construction property of the hypercube (see Figure 16).

Figure 16 

A 2-level HIN using FTBB as basic block [13].

In [14], the Hierarchical Fault-Tolerant Interconnection Network (HFTIN) was proposed. HFTIN uses a different type of FFTB as a basic building block in level one and to use torus at level two, which have a constant degree number. In this network, the FTBB consists of 16 main nodes and 4 spare nodes (see Figure 17). In [20], the performance comparison shows that as the number of faults exceeds six, the HFTIN has a higher probability to recover from faults as compared to HCN architectures.

Figure 17 

A 64 Node FTBB-based HIN [14].

(a)

(b)

(c)

(d)

Figure 18 

(a) Level 0 THIN, (b) level 1 THIN, level 2 THIN, and level 3 THIN [15].

(a)

(b)

Figure 19 

(a) An (2,4,2)-RTTM network and (b) A 4×4 Basic Module [16].

4. Comparisons

The comparisons performed in this section are based on the topological properties of the different hierarchical interconnection networks presented in the previous sections in addition to the Hypercube. Only HINs which are based on the replication technique are considered since they are more regular and practical. HINs that need any configuration parameters are also not included in our comparisons. The included HINs are HIN, dBCube(𝑐,𝑑), HHC, HCN(𝑛,𝑛), HFCube(𝑛,𝑛), FolH, RFH, REH, FloH, and THIN.

Table 1 summarizes the number of nodes, degree and diameter of these hierarchical interconnection networks. Figure 20 shows the relationship of network degree and the number of nodes in the network (network size). The graph shows that the three networks RFH, FloH and THIN possess constant network degree regardless of the network size. The graph also shows that for network size larger than 1024 nodes those same three networks offers the lowest network degree. Among the three networks, the RFH and the THIN networks offer the lowest network degrees. The Graph also shows that the HHC network offer a logarithmic increasing function which results in a lower network degree than the REH network for network size up to 2048 nodes. The Hypercube provides the highest network degree. Figure 21 relates the network size with the diameter. The figure shows that the THIN network possesses the highest network diameter while the HFCube(𝑛,𝑛) provides the lowest network diameter. The diameter of a network is a measure of the network performance in terms of worst-case communication delay.

Table 1 

Topological properties of HINs.

Figure 20 

Comparison in terms of the degrees.

Figure 21 

Comparison in terms of the diameter.

Figure 22 shows the cost (degree × diameter) of networks with respect to its size. The figure shows that the THIN network has the highest network cost while RFH and FloH have the lowest network cost. This is a reflection of the constant low network degree. Figure 23 shows the packing density of networks with respect to its size. The higher the packing density of a network, the smaller the chip area required for its VLSI layout. The figure shows that the HFCube(𝑛,𝑛) has the highest packing density while dBCube(𝑐,𝑑) requires the lowest packing density.

Figure 22 

Comparison in terms of the cost (= degree × diameter).

Figure 23 

Comparison in terms of the packing density (= 𝑁/cost).

5. Conclusion

Hierarchical Interconnection Networks (HINs) provide a design framework of networks with reduced link cost and the efficient use of the locality of communication that exists in parallel applications. In this paper, we reviewed a number of HINs. These networks have been categorized into two main categories based on how they provide fault tolerance. These two categories are (1) the replication technique category and (2) the standby spare node category. A brief description of each hierarchical interconnection network is presented showing the different ways in which HINs are constructed. The paper presented a topological performance comparison among HINs. The topological properties include network degree, diameter, cost, and packing density. The results show that RFH and FloH offer the best network cost while the HFCube(𝑛,𝑛) offer the best packing density, that is, the smallest required chip area for VLSI layout.

Acknowledgments

The authors would like to acknowledge the support of Kuwait University and Al Baha University.