Real-Time Communications in Large-Scale Wireless Networks

Abstract

There is an emerging need for realizing real-time quality of service (QoS) over multihop wireless communications in large-scale wireless networks. The applications can include wireless mesh infrastructure for broadband Internet access supporting multimedia services, visual sensor networks for surveillance, and disaster-relief networks. However, a number of challenges still exist as revealed by recent works, where the dataflow QoS performance such as throughput and end-to-end delay can degrade fast with the number of wireless hops. We propose to use large-scale cognitive networking methods to resolve the wireless multihop challenges. By the cognitive-networking concept, data packets travel along opportunistically available paths in the network with opportunistically available spectrum in every hop. Reliable end-to-end communications can be achieved for real-time services, where we show that (1) dataflow throughput can be independent of any number of wireless hops, (2) end-to-end delay and delay variance increase linearly with the number of wireless hops, and (3) delay variance decreases to zero with higher network density. These results are supported by analysis, simulations, and experiments.

1. Introduction

Large-scale wireless networks, for example, mobile ad hoc networks, wireless mesh networks, as well as wireless sensor networks, have been receiving significant attention in the past few years. Recently, there has been growing interest in realizing real-time quality of service (QoS) over multihop wireless transmissions in large-scale wireless networks. For example, the driving commercial applications include wireless mesh infrastructure for broadband Internet access [1], supporting Voice over Internet Protocols (VoIP) or Multimedia over IP, and wireless sensor network with real-time event detection/reporting, especially in video surveillance, that is, visual sensor networks [2].

Compared to the classical many-to-one (starlike) networks, such as cellular phone networks and WiFi- (IEEE 802.11) [3] enabled wireless local area networks (WLANs), a wireless mesh-based infrastructure for broadband access has the advantage of much lower capital and operational costs and can achieve higher service coverage. As one example, municipal WiFi mesh networks are rolling out in a number of major cities, providing wireless web surfing and email services to general public. Considering the popularity of Internet voice and video, further developments for supporting VoIP within the wireless mesh infrastructure are highly demanded. On the other hand, visual sensor networks appear to be one of the killer applications for wireless sensor networks, since a lot of major cities have installed public-administrated surveillance cameras for various purposes. As of now, those cameras are normally attached by cables. By adopting wireless connections, there will be significant cost reduction in the installation, which can enable much denser deployment. Moreover, related applications also include the disaster-relief wireless networks, with ad hoc formation, where the real-time information delivery can be of life and death importance.

However, many challenges still exist in realizing real-time QoS over multihop wireless networks. It has been well known that dataflow throughput, end-to-end delay, and delay jitter can degrade fast when the number of wireless hops increases [4, 5]. The limitation can be primarily introduced by dynamic network-resource availabilities including both spectrum bandwidth and wireless nodes/radios. Specifically, random spectrum availability is based on wireless fading and interferences prevailing in unlicensed bands (e.g., ISM bands); and random radio availability is due to dynamic traffic load (congestions) and other factors such as radio failure. In traditional wireless networking, the media access control (MAC) layer sets up “logic wired links” over wireless media, which assume predetermined spectrum availability. On top of it, the network layer runs ad hoc routing protocols which assume predetermined radio availability and network topology. It is difficult to have effective QoS models in the traditional network protocol stack [6], when both spectrum bandwidth and radio availabilities cannot be predetermined in large-scale wireless networks.

In order to resolve the wireless multihop limitation, there have been proposals to install multiple radios in one single wireless node [4] and design network routing protocols based on wireless link status, as well as other combined metrics [5]. Although evidence shows that improvements could be obtained, these approaches do not address real-time QoS. Moreover, existing approaches can generally suffer from the lack of scalability. Typically, the complexity of existing protocols can grow fast with the number of wireless nodes neighboring each other (higher network density), since there are a large number of links to be managed and/or tracked on every node. Other research work that investigates real-time QoS in wireless networks could be based on inappropriate assumptions for large-scale wireless networks, such as the assumption on fixed link bandwidth [7].

In this paper, we apply the methods of large-scale cognitive networking [8] to resolve the wireless multihop limitation. The concept of cognitive networking opportunistically utilizes network resources including both spectrum bandwidth and wireless node/radio availability to realize reliable communications in large-scale wireless networks. The adopted network architecture is the Embedded Wireless Interconnect (EWI) [8–11], where wireless linkage is redefined introducing abstract wireless links. By definition, abstract wireless links can be arbitrary functional abstractions of mutual cooperations among proximity wireless nodes, where the categories can include broadcast, unicast, multicast/anycast, and data aggregation, among others. Therefore, wireless link modules are designed as the units for realizing different abstract wireless links at individual wireless nodes. The architecture of EWI is then developed upon two layers, where the bottom wireless link layer supplies a library of wireless link modules; and the upper system layer can organize the modules for achieving effective application programming interface (API).

Conforming to the cognitive networking concept, both the operating spectrum and the participating nodes of an abstract wireless link is opportunistically decided based on their instantaneous availabilities. By the unicast module design in the cognitive networking, data packets can take opportunistically available paths in the network from source to destination, with opportunistically available spectrum in every hop. In comparison, dynamic changes in spectrum and radio availability can create bottlenecks on the routing path of traditional wireless networking, where such bottlenecks are all resolved by the proposed methods. Therefore, reliable end-to-end communications can be achieved, by the opportunistic utilization of network resources. The presented unicast module design is also implemented by cognitive-radio prototyping [12]. In the performance analysis, real-time QoS metrics including throughput, expected end-to-end delay, and delay variance, are studied. The following propositions are obtained.

(i)All the investigated real-time QoS metrics improve with larger network scale, and the complexity on individual nodes remains constant independent of any network scale. It is also shown that the performance dynamics, as indicated by end-to-end delay variance, can diminish to zero with higher network density.(ii)Reliable communications can be achieved over a large number of wireless hops. In principle, the throughput can be independent of the source-to-destination distance; while the expected end-to-end delay, and the delay variance, increase linearly with the source-to-destination distance.(iii)Considering the resource constraints, such as energy consumption or network capacity, we identify that radio transmitting power can be an effective control knob for traffic prioritization in dealing with the tradeoff between achievable QoS performance and resource consumptions.

Therefore, the two major contributions of the paper are the following: (1) the first design & implementation of real-time communications for large-scale wireless networks; (2) the QoS performance analysis of the designed real-time unicast module. In what follows, related works are surveyed in Section 2; the unicast wireless link module design and implementation are described in Section 3; the performance analysis of real-time QoS metrics is presented in Section 4; the simulation and experiment results are reported in Section 5 and 6, respectively; the conclusion is given in Section 7.

2. Related Work

Related works in cognitive networking and the EWI network architecture have previously appeared in [1, 8–12]. As a pilot architectural reference model, EWI was first introduced in the application-specific studies in wireless sensor networks [10, 11]. In [9], recent research works in the cross-layer design of wireless sensor networks are surveyed, where it is suggested that EWI can be an unified design architecture. The concept of large-scale cognitive wireless networks has been presented in [8], where EWI was adopted as the network architecture. In [1, 12], the EWI is further utilized to construct a ubiquitous wireless mesh infrastructure for broadband Internet access.

The unicast design can also be related to existing research works in opportunistic routing [13–17]. In principle, opportunistic routing schemes deal with the codesign of routing and MAC protocols. However, previous research works usually deal with single-hop performance metrics, while the contribution of this paper deals with end-to-end QoS for real-time communications. Moreover, contributions are made in large-scale cognitive networking, with a report of experiment results.

In [13], the protocol selects the next hop relay node by a slotted ACK (acknowledge) mechanism. Having successfully received a data packet, the node calculates a priority level, which is inversely proportional to the distance between the node and the packet destination. The node with the highest priority is then selected distributively as the next hop relay node. In [14, 15], a technique named GeRaF was proposed, where the node forwarding region is divided into smaller sections with different priorities, as decided by the forwarding distances. The performance analysis of GeRaF included the single-hop energy consumption and latency tradeoff, as well as the multihop count number. The study in [16] investigated the architectural aspects of opportunistic routing, where the performance of the component design is analyzed based on single-hop metrics and light-traffic scenario. In [17], the authors discussed the opportunistic relay node selection in a two-hop transmission scenario. The relay-node priority criterion is based on the gain of source-to-relay channels and relay-to-destination channels.

3. Module Design and Implementation

A unicast wireless link is an abstraction of the proximity wireless nodes cooperation for unicasting. Since unicasting usually involves the information delivery from source to destination over multiple hops, the unicast module design decouples the end-to-end network behavior into proximity wireless nodes cooperation. In the following, some elements of the module design and implementation are presented. The module state-diagram is then shown.

3.1. Network Address

Network address is related to the context, subject to a “cost of delivery” criteria. Let 𝑑 and 𝑠 denote the network addresses of the destination node and the source node, respectively. Given the destination 𝑑 of a data packet at one wireless node 𝑛, a local parameter 𝑐_𝑛,𝑑 is assumed obtainable, which indicates the approximate or average cost of delivering the packet to the destination from the node 𝑛 independent of dynamic changes. In large-scale wireless networks, the cost of delivery 𝑐_𝑛,𝑑 is usually a function of the distance from one local node 𝑛 to the destination 𝑑.

In location-centric networks (e.g., [10]), where wireless nodes are aware of their own locations, for example, by global position systems (GPS) or triangulation estimations, the network address of one node 𝑛 is solely decided by the node 𝑛 location coordinates 𝐋_𝑛. Given the destination coordinates 𝐋_𝑑 of a packet, 𝑐_𝑛,𝑑=|𝐋_𝑛−𝐋_𝑑| can be readily calculated in geometry, defined as the specific distance.

In data-centric networks (e.g., [18]), the network address can be decided by the application specific data. The cost of delivery 𝑐_𝑛,𝑑 is the application data gradient, which can be assumed as a monotonically increasing function of the distance between 𝑛 and 𝑑.

In a data-collecting or fusion network, for example, wireless sensor networks, the sink (data collector) can broadcast a number of identity advertisement packets, which is thereon flooded in the network, by broadcasting. Every node can count the average smallest number of hops from the sink, on receiving the advertisement packets. The count number can be used as 𝑐_𝑛,𝑑 for one node 𝑛. A similar approach appeared in [19] yet in a more general scenario, where the logic address, that is, a vector of the estimated distance or hop number to a group of anchors, is maintained for every wireless node. New nodes joining in the network can estimate their own logic address by acquiring the address of neighboring nodes.

Although other types of address can be applicable, we use the location address in this paper for analysis, simulations, and experiments.

3.2. Radio Implementation

The terminology of cognitive radio was first suggested in [20] as an ideal-omnipotent radio which can take all the parameters into consideration for user-centric communications. Cognitive radio was later reviewed in [21, 22] as the radio with dynamic spectrum access. In [8], two propositions are further suggested for the cognitive radio for large-scale wireless networks.

(i)The radio can opportunistically sense the spectrum resource, so that the selected spectrum usage will not be interfering with other on-going wireless communications.(ii)The radio can opportunistically poll one or more other proximity radios onto the selected spectrum, so as to realize certain types of local cooperations.

The above two propositions can extend the concept of pure cognitive radio to cognitive networks, which implement both dynamic spectrum access and dynamic radio access. In the network architecture EWI, a wireless node can initiate an abstract wireless link, that is, certain types of local cooperations among a number of proximity wireless nodes. Hence, both the set of wireless nodes and the operating spectrum are opportunistically decided for an abstract wireless link. Further, the initiated link will not be interfering with other wireless communications, in accordance with the first cognitive proposition.

In the test-bed development of the paper, we prototyped an experimental tone-based cognitive radio [8, 12]. Specifically, the radio can access a group of predetermined data channels, where every data channel is also associated with two distinctive frequency tones, that is, one sensing tone and one polling tone. The sensing and polling tones are at distinctive frequency different from the data channel. The radio hardware is therefore composed of two transceivers, which are the tone transceiver and the data transceiver, respectively. On initiating an abstract wireless link, the radio senses for an available channel, with the vacant data channel and the vacant sensing/polling tones. It then broadcasts the polling tone associated to the selected channel, to poll its surrounding nodes. On detecting the rising edge of the polling tone, the surrounding nodes can decide to join in the initiated abstract wireless link based on their autonomous availability. On joining in an abstract wireless link, the radio of the surrounding nodes also broadcasts the sensing tone. As such, both sensing and polling tones protect wireless link modules from spectrum interferences. We note that the use of frequency (busy) tones has also appeared in numerous works of MAC protocols [23, 24]. However, here we have the first implementation over multiple channels, and such implementation is in the context of large-scale cognitive wireless networks.

The unicast wireless link module is programmed on the cognitive radio prototype (shown in Figure 1). The experiment hardware is a stack of two radio boards (TELOSB [25]). The radio board is composed of one IEEE 802.15.4 [26] compatible transceiver, Chipcon CC2420, one TI MSP430 Microcontroller, and the program interface. The top radio board is utilized as the tone transceiver, while the bottom radio board is utilized as the data transceiver. The two are wired up by the digital interface shown in Figure 1. Three independent data channels in the 2.4 GHz band are allotted to the radio platform, while the associated tones are also in the band of 2.4 GHz. The unicast module is programmed in the firmware of TI MSP430.

Figure 1 

Cognitive radio prototype.

3.3. Packet Relaying Process

Under an initiated wireless link, the packet relaying process picks one available relay node and sends the data information from the source or a previous relay node to the new relay node.

Four types of packets are used in the packet relaying process, which are RTS, CTS, DATA, and ACK, respectively. RTS, CTS, and ACK are control packets, while DATA is the data information unit. Let node 𝑛_𝑝 denote the relay node at the hop 𝑝, the next hop relay node 𝑛_𝑝+1 is found by the following procedure, which incurs local wireless nodes cooperation in proximity, that is, a unicast wireless link.

First, the node 𝑛_𝑝 broadcasts a RTS, including the module type (unicast), the destination address 𝑑, and the self-address 𝑛_𝑝. Upon receiving the RTS at one node 𝑛, if the condition 𝑐_𝑛,𝑑<𝑐_{𝑛𝑝,𝑑} is satisfied, the node 𝑛 initializes a timer, with the timeout period as 𝑇_𝑑(𝑛_𝑝,𝑛,𝑑). 𝑇_𝑑(𝑛_𝑝,𝑛,𝑑) is inversely proportional to 𝑐_{𝑛𝑝,𝑑}−𝑐_𝑛,𝑑, which is determined by the specific physical radio technology, for example,𝑇_𝑑𝑛^𝑝,𝑛,𝑑=𝐶⁰𝑐_{𝑛𝑝,𝑑}−𝑐_𝑛,𝑑+SIFS,(1)where 𝐶₀ is a constant, and 𝑛≠𝑑. SIFS is the smallest (minimal) inter frame space (delay constant), which is composed of the module processing time and the transceiver RX/TX switch time. In our current test-bed implementation, 𝑇_𝑑(𝑛_𝑝,𝑛,𝑑) is also slotted according to the minimal carrier-sensing time. Note that the condition 𝑐_𝑛,𝑑<𝑐_{𝑛𝑝,𝑑} is enforced, so that the cost of delivery to the destination is always descending which prevents any loop.

The node 𝑛 then backs off and monitors the data channel for the period 𝑇_𝑑(𝑛_𝑝,𝑛,𝑑). If the data channel is free during that period, the node 𝑛 replies with a CTS, declaring itself as the next hop relay node 𝑛_𝑝+1. Otherwise, the node 𝑛 quits the unicast module. As such, the node with the smallest cost 𝑐_𝑛,𝑑 should be elected as the next hop relay node 𝑛_𝑝+1. After having received the CTS, the node 𝑛_𝑝 transmits the DATA packet, and after receiving the DATA packet, the node 𝑛_𝑝+1 replies an ACK, completing the round of relaying. In the case that more than one node send out their CTS simultaneously, the transmissions will collide on some specific bits, for example, the node address, where the CTS packets are different. Since this collision can be detected by certain mechanism, such as cyclic redundancy check (CRC), at 𝑛_𝑝, the node 𝑛_𝑝 can use an appended procedure to differentiate one node as 𝑛_𝑝+1, either before or after the DATA transmission.

The described mechanism is illustrated in Figure 2. In the last hop, the destination 𝑑 is always assigned with the minimal delay, that is, 𝑇_𝑑(𝑛_𝑝,𝑑,𝑑)=SIFS.

Figure 2 

Packet relaying mechanism.

3.4. Module State Diagram

The state diagram of a single unicast module in the wireless link layer is shown in Figure 3, which can give a summary of the implementation. The definitions of the states and the associated transferring branches can be self-explainable, in accordance with previous descriptions.

Figure 3 

Wireless link layer state diagram.

The wireless link layer stays in the IDLE state, when no module is involved. It can initiate a unicast module, for example, on receiving the command from the system layer, so as to send certain amount of information (i.e., contained in one DATA packet) to a specified destination. In the IDLE state, the wireless link layer also monitors the set of polling tones, which are associated to the predetermined group of data channels. On detecting the rising edge of one polling tone, it sends one module request to the system layer. Upon approval, the wireless link layer listens to the RTS packet on the data channel, which is associated with the detected polling tone. Otherwise, it goes back to the IDLE state. Furthermore, after having received one DATA packet, the unicast module automatically relays it, if the current node is not the destination.

Table 1 shows an example set of primitive functions on the interface between the system layer and the wireless link layer, as related to the unicast module. The parameter Priority in Table 1 can be used for specifying the traffic class, which will be explored further in Section 4.5.3.

Table 1 

Exemplary primitive functions related to the unicast module.

3.5. Discussion

By opportunistically dealing with network resources including both spectrum bandwidth and radio availability, data packets take opportunistically available paths with opportunistically available spectrum in every hop. Since the unicast module operates opportunistically in proximity wireless nodes, the end-to-end performance of real-time QoS, can be statistically assured, adding up the sequential proximity operations. This discussion can provide an intuitive explanation of the performance analysis results in Section 4.

Specifically, random spectrum availability is handled by the cognitive radio implementation, where a source or relay node tries to find an available data channel for initiating the unicast wireless link. Important aspects of random node/radio availability are as follows: the node deployment, mobility, and traffic congestion. In general, it indicates that the node 𝑛_𝑝, that is, the 𝑝th hop relay, would be uncertain about the availability of the next hop relay node, before the RTS probing. The RTS/CTS exchange in the module design opportunistically finds the next hop relay, in a group of available candidates.

In traditional computer networks, dynamic traffic load is limited by network congestion control. The network layer drops overflowed packets by queueing management, which is then detected by the transport layer. Such methods have been known to incur problems in wireless networks [27]. By the real-time unicast module, the paradigm of the classical “network queueing” could be transferred to “queueing in network.” In an individual relay node, a DATA packet is forwarded automatically without lengthy buffering in one local queue. Sequential DATA packets nominally take opportunistically decided paths from the source to the destination, which are buffered in the network, instead of any predetermined nodes.

Therefore, it can be intuitive to conceive that the QoS performance metrics should improve with larger network scale, since the network density contributes to the diversity (or more radio resource) that can be opportunistically exploited. This agrees with other theoretical results, for example, in [28], where it is stated that summed network transport capacity should increase with network scale, on the order of Θ(√𝑁), where 𝑁 is the number of nodes.

Moreover, in the proposed method, any node does not need to consider or know every other possible node as a candidate for the next hop. In the formation of abstract wireless linkage, the participating nodes are decided by their autonomous availability. The unicast module design also assures that there would only be one RTS response in most cases by the carrier-sensing mechanism. Therefore, the module complexity at individual nodes remains constant, independent of the network scale.

4. Performance Analysis of Real-Time QoS

The performance analysis mathematically quantifies the major propositions in the paper. The results can also provide useful network planning guidance for deployments.

4.1. Objectives and Approaches

In supporting real-time QoS, the metrics of throughput, expected end-to-end delay, and delay variance, are studied for the unicast wireless link module. Two types of resource consumptions are considered, which are network capacity and network energy consumption. Therefore, we centrally explore the following three questions. (1) How do the QoS metrics change with network scale? (2) How do the QoS metrics change with the source-to-destination distance? (3) How to control the resource allocation, so as to achieve the QoS requirements?

Given the source 𝑠 and the destination 𝑑 of a unicast dataflow, let 𝑙=|𝐋_𝑠−𝐋_𝑑| denote the source-to-destination distance. Analytical results are obtained under the asymptotic condition of long-range unicasting, 𝑙→∞, where the justification can be that long-range unicasting presents the worst-case scenario of the network performance under practical interests. On the other hand, concise analytical formulation of the QoS metrics can be obtained under this long-range asymptotic condition, which can provide analytical insights to previous questions.

4.2. Analytical Models

In the performance analysis, models about node distribution, wireless channel, power consumption, bandwidth availability, and the IDLE probability are adopted.

4.2.1. Node Distribution Model

The node distribution is modeled as a two-dimensional (2D) Poisson process [29], with the node density 𝜆. That is, given an area 𝐀 of the size |𝐀| in the field, the number of nodes in the area follows Poisson distribution with the parameter 𝜆|𝐀|. The Poisson modeling can be typical for random node distribution and/or random mobility.

4.2.2. Wireless Channel Model

Given an arbitrary transmitting node 𝑛, and a receiving node 𝑚, the successful transmission probability, in general, is a function of the transmitting power 𝑃_𝑡, the radio data rate 𝑅, and the distance 𝜁=|𝐋_𝑚−𝐋_𝑛|. We denote this function as 𝑓(𝑃_𝑡,𝑅,𝜁).

For example, if small-scale Rayleigh fading [30] is assumed, the channel model can be given by𝑓(𝑃_𝑡,𝑅,𝜁)=Prob𝑃^𝑡⋅𝐺⋅𝜉𝑁₀⋅𝑅⋅𝜁^𝛼≥𝐵=𝑒^{−(𝑁0⋅𝑅⋅𝜁𝛼⋅𝐵)/(𝑃𝑡⋅𝐺)}.(2)In (2), 𝐺 is a propagation-loss constant; 𝑁₀ is the receiver-noise power spectrum density; 𝛼 is the path loss component in wireless channel [30]; 𝐵 is a threshold constant representing the receiver sensitivity; and 𝜉 is a unit-mean exponential random variable. This model will also be utilized to obtain the numerical results in Section 5.

4.2.3. Power Consumption Model

Referring to the state diagram in Figure 3, the node power consumption at the IDLE state is denoted by 𝑃_𝐼, that is, the low-power monitoring of the polling tones; the power consumption at the states of  “Send DATA”, “Send RTS”, “Send CTS”, and “Send ACK”, is modeled by𝑃_𝑆+(1+𝛽)⋅𝑃_𝑡,(3)where 𝑃_𝑆 is the transmitter circuits power consumption; 𝛽 is a constant decided by the RF power amplifier efficiency [31]; and 𝑃_𝑡 is the transmitting power as defined perviously. The node power consumption in other states or time is modeled by a constant 𝑃_𝑅, which denotes the receiving (or idle listening) power consumption.

4.2.4. Bandwidth Availability and the IDLE Probability

When initiating a unicast module, we assume that the node can always find an available channel by the cognitive radio without significant delay. The IDLE probability is the probability that a given node is in the IDLE state (shown in Figure 3), and can be engaged in a unicast wireless link being initiated by other nodes around it. We assume that there is a fixed (or lower bounded) IDLE probability “𝜌”, in the analysis.

With limited network capacity and arbitrary traffic load, these two assumptions need to be ensured by an appropriate call admission control (CAC) mechanism. On the other hand, given analogous experiences of the popular video streaming on peer-to-peer overlay networks, it could also be appropriate to assume no CAC mechanism, while the traffic volume demanding can always fall below the planned (or unplanned) network capacity. In large-scale wireless networks, the capacity could be of relatively cheap resources, given abundant wireless nodes and the use of unlicensed band, while the traffic load is not necessarily related to the number of wireless nodes. These considerations give the optimization formulations under different types of resource consumptions in Section 4.5.

4.2.5. Other Parameters and Notations

The packet lengths (bit) of RTS, CTS, DATA, and ACK are denoted by 𝐿_𝑅, 𝐿_𝐶, 𝐿_𝐷, and 𝐿_𝐴, respectively. 𝑇_𝑆 denotes the time delay in the “channel sensing” state in Figure 3, and 𝑇_𝐵 denotes the time delay in the “Backoff 𝑇_𝑑” state. We also use the notations 𝐸(⋅) and 𝜎²(⋅) to denote the mean and the variance of a random variable.

4.3. Iterative Performance Analysis

“Iterative performance” can be similar to “single-hop performance.” However, with the notation of “iterative performance,” we also consider the scenario where the next hop relay node is not found in the transferring branch from “Send RTS” to “channel sensing” of Figure 3. The reason resides in the random node availability, for example, no potential relay candidate is in the IDLE states. Due to this difference, the terminology “iteration” instead of “hop” is used in the following analysis, and “iterative delay” indicates the time delay of an iteration, whereas “iterative energy consumption” indicates the network energy consumption in one iteration.

Particularly, consider that there are totally 𝐼 iterations from the source 𝑠 to the destination 𝑑. Let {𝑛_𝑖∣1≤𝑖≤𝐼} denote the revelent set of nodes, which initiates the unicast wireless link. Obviously, there is 𝑛₁=𝑠, and 𝑛_𝐼 sends the DATA packet directly to the destination 𝑑. The iteration number 𝐼 is therefore a random variable.

4.3.1. Iterative Delay

Let 𝜏_𝑖 denote the iterative delay at an iteration 𝑖. (Other possible models are also discussed in Appendix C, which contribute to the same end-to-end performance analysis results.) If the DATA transmission is completed in the iteration, 𝜏_𝑖 can be calculated by the formula (𝐿_𝑅+𝐿_𝐶+𝐿_𝐷+𝐿_𝐴)/𝑅+𝑇_𝑆+𝑇_𝐵. Otherwise, if the DATA transmission is uncompleted since no relay is found, 𝜏_𝑖 can be given by 𝐿_𝑅/𝑅+𝑇_𝑆+𝑇_𝐵. Obviously, for the last iteration (𝑖=𝐼), the DATA transmission is completed.

When 𝑖<𝐼, the probability of uncompleted DATA transmission depends on the relay candidates availability on the forwarding plane. Due to the Poisson node distribution model with the density 𝜆, and the node IDLE probability 𝜌, the distribution of relay candidates conforms to an inhomogeneous Poisson process with the density function 𝜆⋅𝜌⋅𝑓(𝑃_𝑡,𝑅,𝜁) on the forwarding plane, where 𝜁 indicates the distance to the node 𝑛_𝑖. Therefore, the “uncompleted” probability is decided by the “equal-zero” probability of a Poisson random variable with the mean 𝜆⋅𝜌⋅Ω, which is given by 𝑒^{−𝜆𝜌Ω}. Ω is always positive, and is given by the integration of the channel model 𝑓(⋅,⋅,⋅) on the forwarding plane:Ω=^∞0^∞−∞𝑓(𝑃_𝑡,𝑅,√𝑥²+𝑦²)𝑑𝑥𝑑𝑦.(4)

According to the above analysis, {𝜏_𝑖∣1≤𝑖<𝐼} can be modeled as a set of random variables with identical distributions, where the expectation 𝐸(𝜏_𝑖) is𝐸(𝜏_𝑖)=𝐿^𝑅𝑅+𝑇^𝑆+𝑇_𝐵+𝐿^𝐶+𝐿_𝐷+𝐿_𝐴𝑅⋅1−𝑒^{−𝜆𝜌Ω}.(5)

4.3.2. Iterative Energy Consumption

Let 𝜖_𝑖 denote the iterative energy consumption at an iteration 𝑖. (Other possible models are also discussed in Appendix C, which contribute to the same end-to-end performance analysis results.) If the DATA transmission is not completed in the iteration, the energy consumption of the node 𝑛_𝑖 is (𝑇_𝑆+𝑇_𝐵)⋅𝑃_𝑅+𝐿_𝑅/𝑅⋅[𝑃_𝑆+(1+𝛽)𝑃_𝑡], according to the described models. The energy consumption of one relay candidate is given by 𝑇_𝑆⋅𝑃_𝐼+𝐿_𝑅/𝑅⋅𝑃_𝑅. The number of the relay candidates around 𝑛_𝑖, which receive the RTS, is a Poisson random variable, with the mean value 𝜆⋅𝜌⋅Ω. Here Ω can be still given by (4), but is integrated on the backward plane, that is, ∫^0−∞∫^∞−∞𝑓(𝑃_𝑡,𝑅,√𝑥²+𝑦²)𝑑𝑥𝑑𝑦, instead of the forwarding plane. Therefore, the expected iterative energy consumption under uncompleted DATA transmission is the summation: (𝑇_𝑆+𝑇_𝐵)⋅𝑃_𝑅+𝐿_𝑅/𝑅⋅[𝑃_𝑆+(1+𝛽)𝑃_𝑡]+𝜆⋅𝜌⋅Ω⋅(𝑇_𝑆⋅𝑃_𝐼+𝐿_𝑅/𝑅⋅𝑃_𝑅).

If the DATA transmission is completed in the iteration, the energy consumption of the node 𝑛_𝑖 is (𝑇_𝑆+𝑇_𝐵)⋅𝑃_𝑅+(𝐿_𝑅+𝐿_𝐷)/𝑅⋅[𝑃_𝑆+(1+𝛽)𝑃_𝑡]+(𝐿_𝐶+𝐿_𝐴)/𝑅⋅𝑃_𝑅. The energy consumption of one relay candidate, which is not elected as the next node 𝑛_𝑖+1, is still 𝑇_𝑆⋅𝑃_𝐼+𝐿_𝑅/𝑅⋅𝑃_𝑅. The number of the relay candidates is a Poisson random variable with the mean value 2𝜆⋅𝜌⋅Ω, including both the forward and the backward planes. On the node 𝑛_𝑖+1, the additional energy consumption is 𝐿_𝐷/𝑅⋅𝑃_𝑅+(𝐿_𝐶+𝐿_𝐴)/𝑅⋅[𝑃_𝑆+(1+𝛽)𝑃_𝑡]. Therefore, the expected iterative network energy consumption under completed DATA transmission is (𝑇_𝑆+𝑇_𝐵+(𝐿_𝐶+𝐿_𝐷+𝐿_𝐴)/𝑅)⋅𝑃_𝑅+(𝐿_𝑅+𝐿_𝐶+𝐿_𝐷+𝐿_𝐴)/𝑅⋅[𝑃_𝑆+(1+𝛽)𝑃_𝑡]+2𝜆⋅𝜌⋅Ω⋅(𝑇_𝑆⋅𝑃_𝐼+𝐿_𝑅/𝑅⋅𝑃_𝑅).

Since the DATA transmission is always completed in the last iteration, 𝜖_𝐼 is directly obtained from the previous discussion. When 𝑖<𝐼, similar to the previous analysis of 𝜏_𝑖, the probability of uncompleted DATA transmission is decided by 𝑒^{−𝜆𝜌Ω}. Therefore, {𝜖_𝑖∣1≤𝑖<𝐼} can be modeled as a set of random variables with identical distributions, where the expectation 𝐸(𝜖_𝑖) is𝐸(𝜖_𝑖)=𝑇_𝑆+𝑇_𝐵⋅𝑃^𝑅+𝐿^𝑅𝑅⋅𝑃^𝑆+𝜆⋅𝜌⋅Ω⋅𝑇_𝑆+(1+𝛽)𝑃_𝑡⋅𝑃_𝐼+𝐿^𝑅⋅{𝐿_𝐶𝑅⋅𝑃^𝑅+1−𝑒^{−𝜆𝜌Ω}+𝐿_𝐷+𝐿_𝐴𝑅⋅𝑃^𝑅+𝜆⋅𝜌⋅Ω⋅𝑇_𝑆+𝑃_𝑆+(1+𝛽)𝑃_𝑡⋅𝑃_𝐼+𝐿^𝑅𝑅⋅𝑃^𝑅}.(6)

4.3.3. Number of Iterations

Given all the parameters, the total iteration number 𝐼 is a random variable decided by the source-to-destination distance 𝑙. It is shown in Appendix A that the mean value 𝐸(𝐼) and the variance 𝜎²(𝐼) are𝐸(𝐼)=𝑙𝐸(Λ)+𝑜√𝜎𝑙,(7)₂(𝐼)=𝑙⋅𝜎²(Λ)𝐸(Λ)³+𝑜(𝑙),(8)respectively.

In the above, the random variable Λ generally indicates the forwarding distance of one single iteration. Λ is defined with the pdf (probability density function) 𝑝_Λ(𝑥):𝑝_Λ(𝑥)=⎧⎪⎪⎨⎪⎪⎩𝜆𝜌⋅𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑒⋅𝑔(𝑥),𝑥>0,_{−𝜆𝜌Ω}0,⋅𝛿(0),𝑥𝑥=<00,,(9)where𝑔(𝑢)=^∞−∞𝑓𝑃_𝑡,𝑅,√𝑢²+𝑣²𝑑𝑣.(10)

Proved in Appendix B, we point out that the expectation 𝐸(Λ) is an increasing function of the node density 𝜆, but is upper bounded, as decided by the channel model 𝑓(⋅,⋅,⋅). Furthermore, the variance 𝜎²(Λ) decreases to zero with higher 𝜆. Given (7) and (8), both 𝐸(𝐼) and 𝜎²(𝐼) can decrease with higher node density 𝜆. These relations can also be intuitively understood, which will be exploited for interpreting the end-to-end performance analysis results next.

4.4. End-to-End Performance Analysis

Analytical results are presented for the maximal throughput Φ, the expected end-to-end delay Υ, the end-to-end delay variance Θ, and the expected network energy consumption per DATA packet Ξ, of a dataflow from the source 𝑠 to the destination 𝑑, under the long-range asymptotic condition 𝑙→∞. The discussion shows how these metrics change with the source-to-destination distance 𝑙, as well as the network density 𝜆.

4.4.1. Throughput

The maximal throughput Φ is decided by how many DATA packets can be sent out from the source 𝑠 to the destination 𝑑 in a period of time, subject to the condition that this time period is much larger than the packet traveling time. According to previous analysis in Section 4.3.1, the probability of the completed DATA transmission in an iteration is 1−𝑒^{−𝜆𝜌Ω}. Therefore, the expected number of iterations initiated by 𝑠 for one DATA packet is 1/(1−𝑒^{−𝜆𝜌Ω}), and the associated time expectation is (𝐿_𝑅/𝑅+𝑇_𝑆+𝑇_𝐵)/(1−𝑒^{−𝜆𝜌Ω})+(𝐿_𝐶+𝐿_𝐷+𝐿_𝐴)/𝑅.

Given the DATA packet length 𝐿_𝐷, the maximal throughput is thus obtained asΦ=𝐿^𝐷⋅𝑅𝐿^𝑅+𝑇_𝑆+𝑇_𝐵⋅𝑅/1−𝑒^{−𝜆𝜌Ω}+𝐿^𝐶+𝐿_𝐷+𝐿_𝐴.(11)

It is shown that the maximal throughput Φ can be independent of the source-to-destination distance 𝑙. It also increases monotonically with the node density 𝜆, approaching the limit decided by the radio data rate 𝑅 and the overhead ratio.

4.4.2. Expected End-to-End Delay

The end-to-end delay is given by ∑^𝐼𝑖=1𝜏_𝑖. The expected end-to-end delay Υ isΥ=^(a)𝐸(^𝐼_𝑖=1=_(b)𝜏_𝑖)𝐸(𝐼)−1⋅𝐸𝜏¹+𝜏^𝐼=^(c)𝑙⋅𝐿^𝑅+𝑇_𝑆+𝑇_𝐵⋅𝑅+𝐿^𝐶+𝐿_𝐷+𝐿_𝐴⋅1−𝑒^{−𝜆𝜌Ω}+𝑜√𝑅⋅𝐸(Λ)𝑙,(12)where (a) is given by the definition; (b) is due to the fact that {𝜏_𝑖∣1≤𝑖<𝐼} are of identical probability distribution [32]; and (c) is obtained directly from (5) and (7).

Therefore, the expected end-to-end delay Υ increases linearly with the source-to-destination distance 𝑙. It also decreases with the node density 𝜆, approaching a bounded limit.

4.4.3. Delay Variance

Let 𝐼_𝑐 denote the number of the iterations with completed DATA transmission, that is, the number of wireless hops, and let 𝐼_𝑢 denote the number of the iterations with uncompleted DATA transmission. 𝐼_𝑐 and 𝐼_𝑢 are independent random variables, and 𝐼=𝐼_𝑐+𝐼_𝑢. The delay variance Θ is calculated by the following:Θ=^(a)𝜎²(^𝐼_𝑖=1=_(b)𝜏_𝑖)𝜎²(𝐼_𝑐)⋅𝐿^𝑅+𝐿_𝐶+𝐿_𝐷+𝐿_𝐴𝑅+𝑇^𝑆+𝑇_𝐵²+𝜎²(𝐼_𝑢)⋅𝐿^𝑅𝑅+𝑇^𝑆+𝑇_𝐵²=^(c)𝜎²(𝐼_𝑐)⋅[𝐿^𝐶+𝐿_𝐷+𝐿_𝐴𝑅²+2⋅𝑇_𝑆+𝑇_𝐵+𝐿^𝑅⋅𝐿_𝐶𝑅+𝐿_𝐷+𝐿_𝐴𝑅]+𝜎²(𝐼)⋅𝐿^𝑅𝑅+𝑇^𝑆+𝑇_𝐵²=^(d)𝑙⋅𝜎²(Λ)𝐸(Λ)³⋅{1−𝑒^{−𝜆𝜌Ω}²⋅[𝐿^𝐶+𝐿_𝐷+𝐿_𝐴𝑅²+2⋅𝑇_𝑆+𝑇_𝐵+𝐿^𝑅⋅𝐿_𝐶𝑅+𝐿_𝐷+𝐿_𝐴𝑅]+𝐿𝑅^𝑅+𝑇^𝑆−𝑙⋅𝑒_{−𝜆𝜌Ω}+𝑇_𝐵²}⋅1−𝑒^{−𝜆𝜌Ω}𝐸(Λ)⋅[𝐿^𝐶+𝐿_𝐷+𝐿_𝐴𝑅²+2⋅𝑇_𝑆+𝑇_𝐵+𝐿^𝑅⋅𝐿_𝐶𝑅+𝐿_𝐷+𝐿_𝐴𝑅]+𝑜(𝑙),(13)where (a) is given by the definition; (b) is due to the analysis in Section 4.3.1, that is, the iterative delay under completed and uncompleted transmissions respectively; (c) is due to the fact that 𝜎²(𝐼)=𝜎²(𝐼_𝑐)+𝜎²(𝐼_𝑢); (d) is obtained from the result of 𝜎²(𝐼) in (8), as well as the 𝜎²(𝐼_𝑐) derived in (A.4) of Appendix A.

Therefore, the end-to-end delay variance Θ increases linearly with the source-to-destination distance 𝑙. Θ also diminishes to zero with the network density 𝜆, since the component 𝜎²(Λ) also decreases to zero with higher 𝜆.

4.4.4. Network Energy Consumption

The expected network energy consumption for one DATA packet Ξ isΞ=^(a)𝐸(^𝐼_𝑖=1=_(b)𝜖_𝑖)𝐸(𝐼)−1⋅𝐸𝜖¹+𝜖^𝐼=^(c)𝑙⋅𝐸(𝜖¹)𝐸(Λ)+𝑜√𝑙,(14)where (a) and (b) are given by the definition and the fact that {𝜖_𝑖∣1≤𝑖<𝐼} are of identical probability distributions [32]; (c) is obtained directly from (7), and the component 𝜖₁ can be replaced by the result in (6). Therefore, Ξ also increases linearly with the source-to-destination distance 𝑙.

4.5. Optimizations under QoS Constraints

Consider that the dataflow has certain QoS requirements on throughput, expected end-to-end delay, and delay variance, for example, Φ≥Φ_th, Υ≤Υ_th, and Θ≤Θ_th. Here, Φ_th, Υ_th, and Θ_th are the threshold constants representing those QoS constraints, respectively. The optimizations aim to minimize the resource consumption, subject to these QoS constraints.

Two types of resource consumptions, including network energy consumption and network capacity, are considered here, which lead to different optimization formulations. As pointed out in Section 4.2.4, these two formulations can be representing different networking scenarios. In capacity limited networks, every node can have cabled power supply. The number of nodes is therefore limited by engineering costs, and the possible use of long-range radio also limits the available bandwidth. On the other hand, in energy limited networks, the nodes can be powered by energy scavenging, for example, solar cells. Therefore, the number of wireless nodes can be large, which can also use short-range radio with vast unlicensed bandwidth. These contribute to large network capacity. Other works with similar considerations have been discussed in [33].

4.5.1. Capacity Limited Networks

The objective of the optimization is to minimize the transmitting power 𝑃_𝑡, so as to limit the occupied geographic area of an unicast link. In accordance with the results in [28], this approach can save the use of wireless network capacity. Therefore, the optimization formulation ismin𝑃_𝑡subjectto:Φ≥Φ_th;Υ≤Υ_th;Θ≤Θ_th.(15)

4.5.2. Energy Limited Networks

The objective of the optimization is to minimize the expected network energy consumption per DATA packet Ξ:smubinjeΞctto:Φ≥Φ_th;Υ≤Υ_th;Θ≤Θ_th.(16)

4.5.3. Solutions and Traffic Priority

In order to solve the optimizations in (15) and (16), the transmitting power 𝑃_𝑡 can be an ideal control knob (optimization variable), for the resource allocation. Both (15) and (16) can be found as convex optimization problems over 𝑃_𝑡, where efficient solutions are guaranteed [34].

Specifically, as shown in (11), Φ monotonically increases with 𝑃_𝑡, since Ω (defined in (4)) monotonically increases with 𝑃_𝑡, provided by any realistic channel model 𝑓(⋅,⋅,⋅). As shown in (12), Υ monotonically decreases with 𝑃_𝑡, since 𝐸(Λ) monotonically increases with 𝑃_𝑡 (proved in Appendix B). Θ also monotonically decreases with 𝑃_𝑡 in (13), since 𝜎²(Λ) monotonically decreases with 𝑃_𝑡 (proved in Appendix B). Moreover, as shown in (14), Ξ can be verified as a convex function of 𝑃_𝑡, where both 𝐸(𝜖₁) and 𝐸(Λ) monotonically increase with 𝑃_𝑡. Due to the described monotonicity, the convexity of the optimizations in (15) and (16) can then be obtained, over the control knob 𝑃_𝑡.

Therefore, we suggest that the transmitting power 𝑃_𝑡 can provide an effective control knob for the tradeoff  between the resource consumptions and the real-time QoS requirements. In fact, different levels of 𝑃_𝑡 can be configured for different classes of traffics, for example, voice, video (streaming), or data, which then decide the unicasting “priority”.

Further investigation may also include the radio data rate 𝑅, so as to generate the problem of joint power and rate control. Unfortunately, the convexity over 𝑅 cannot be obtained for a general channel propagation model 𝑓(⋅,⋅,⋅). Future studies may address the problem on some specific models.

4.6. Summary

In summary, the analytical results indicate that all the investigated real-time QoS metrics can improve with larger network scale. In particular, the maximal throughput Φ increases with the network density 𝜆 to a predetermined bound, while the expected end-to-end delay Υ decreases with 𝜆, approaching a predetermined bound, as decided by other related parameters. On the other hand, the delay variance Θ falls to zero with higher network density 𝜆, which indicates that the network performance can be made arbitrarily stable, simply by dropping more nodes in the network. For supporting long-range communications over a relatively large number of wireless hops, the analytical results show that the maximal throughput Φ can be independent of the source-to-destination distance, while both the expected delay Υ and the delay variance Θ increase linearly with the source-to-destination distance. Furthermore, it is identified that the radio transmitting power 𝑃_𝑡 can be an ideal control knob for the tradeoff between the real-time QoS requirements and the resource consumptions, for example, energy consumption or network capacity.

We consider that all these propositions could be intuitively conceivable for large-scale wireless networks. Since the analytical results are obtained under the long-range asymptotic condition 𝑙→∞, they may present the worst-case scenario analysis for practical considerations. In the next section, the simulation results are also provided, showing how these metrics converge.

5. Simulation Results

Some fixed parameters in the simulations are listed in Table 2. In particular, the radio transmission parameters are conforming to the developed test-bed, while others, for example, power consumption and channel model parameters, are typical for indoor short-range radios. For example, under these parameters, the radio range is about 10-15 meters, when the transmitting power 𝑃_𝑡 is 1mW.

Table 2 

Parameters in the simulations.

Given the source and the destination nodes, separated by the distance 𝑙, random Poisson points are generated, for every iteration. The generation of the Poisson points conforms to the density 𝜆⋅𝜌 on the 2D plane, which represent the available wireless nodes. 1000 DATA packets are then “sent out” consecutively from the source to the destination, according to the described real-time unicast module. The end-to-end delay instance of every DATA packet is then recorded, from which the simulated end-to-end delay expectation and the simulated delay variance are estimated by standard statistical methods. The simulated throughput is also obtained, by dividing the data bulk size 1000⋅𝐿_𝐷 with the time consumption of sending out the 1000 DATA packets at the source. In the following figures, we also compare the simulation results to the analytical results side-by-side, where the lower-order terms, for example, 𝑜(√𝑙) or 𝑜(𝑙), are neglected in plotting the analytical results.

Figures 4, 5, and 6 show how the QoS metrics change with the source-to-destination distance 𝑙, where the node density 𝜆 is fixed at 0.1/m². Then Figures 7, 8, and 9 show how the QoS metrics change with the network density 𝜆, where the source-to-destination 𝑙 is fixed at 160m. The number of wireless hops in the simulations can be up to 20. All the obtained simulation results match well with the analytical results, as well as the main propositions given in Section 4.6. Especially, Figures 4, 5, and 6 show that close-matching also exists in short-range unicasting, although the analytical results were obtained under long-range asymptotic approximation.

Figure 4 

Throughput versus source-to-destination distance 𝑙 (𝜆=0.1/m2).

Figure 5 

Expected delay versus source-to-destination distance 𝑙 (𝜆=0.1/m2).

Figure 6 

Delay variance versus source-to-destination distance 𝑙 (𝜆=0.1/m2).

Figure 7 

Throughput versus network density 𝜆 (𝑙=160m).

Figure 8 

Expected delay versus network density 𝜆 (𝑙=160m).

Figure 9 

Delay variance versus network density 𝜆 (𝑙=160m).

Moreover, Figure 10 shows how the expected network energy consumption per DATA packet changes with the transmitting power 𝑃_𝑡, which is observed as a convex function. It is also observed in Figure 10 that the expected network energy consumption increases linearly with 𝑙, and it decreases with 𝜆 in the simulation. These simulation results conform to previous performance analysis.

Figure 10 

Network energy consumption versus transmitting power 𝑃𝑡.

6. Experiment Results

In the test-bed (Figure 11), the source and the destination nodes are connected to a laptop computer for the purpose of collecting experiment records. The wireless nodes are based on the cognitive radio prototype (Figure 1), with a stack of two radio boards. The cognitive radio can opportunistically access 3 data channels in 2.4 GHz band, as described in Section 3.2, where each channel has an air-interface data rate 250kbps as defined in IEEE 802.15.4 [26].

Figure 11 

A snapshot of the test-bed.

The real-time unicast module is implemented in the cognitive radio prototype, with preconfigured address. The packet length parameters, that is, 𝐿_𝑅, 𝐿_𝐶, 𝐿_𝐷, 𝐿_𝐴, are about the same as those listed in Table 2. The node transmitting power is however fixed at −15dBm. The experiment results on throughput, end-to-end delay expectation, and delay variance, are obtained by the following methods. (1) Throughput is calculated at the source, where the laptop computer forwards 1000 DATA packets to the source node. Since the source node accepts a new DATA packet from the computer only after the previous one has been sent out, the computer can count the total time expenditure in sending the 1000 DATA packets. The measured throughput is obtained by dividing the bulk size with the time expenditure. (2) On receiving a DATA packet, the destination also forwards the DATA packet to the laptop computer, where the end-to-end delay instance of every DATA packet is calculated at the computer by the difference between the packet departure and arrival time. The measured end-to-end delay expectation and the delay variance are then obtained by standard statistical methods. Since the operating system running at the laptop computer (Windows XP Professional Edition) is by no means real-time, we tried to minimize the computer running threads in the experiments.

The experiments were conducted in a standard office environment, where external interferences in the utilized 2.4 GHz band can be intensive, for example, from adjacent WLAN hot spots. Two sets of experiments are illustrated in Figure 12. In Set-I, the nodes are placed in a straight line, such that the four configurations correspond approximately to one-hop to four-hop communications. For example, in the second configuration of Set-I, the node placement is such that the chance of direct transmission from 𝑠 to 𝑑 is small. In Set-II, more relay nodes are added to every hop being illustrated in Figure 12. The purpose of the two-set experiments is to compare how the designed unicast module can improve the test-bed QoS performance. In fact, Set-I experiments can be comparable to traditional wireless networking where a routing path is predetermined with best available spectrum in every hop. However, in Set-II experiments, data is forwarded from 𝑠 to 𝑑 opportunistically without any predetermined path.

Figure 12 

Illustration of the two sets of experiments.

Figures 13, 14, and 15 show the measurement results of the two-set experiments on throughput, expected end-to-end delay, and delay variance, respectively. Evidently, the results of Set-II are strictly better than those of Set-I, beyond the first configuration. In Figure 13, the throughput of Set-I is reduced by more than 50% from the first to the second configuration, that is, from one-hop to two-hop, due to the impact of half-duplex radios. A comparable reduction of throughput is not observed in Set-II. In general, the throughput of Set-II changes smoothly with different configurations (number of wireless hops), where the limited reduction can be due to the limited number of nodes. In Figures 14 and 15, it is further observed that the expected end-to-end delay and the delay variance increase linearly with the number of hops (configuration index) in Set-II experiments; whereas they tend to increase much faster in Set-I experiments.

Figure 13 

Measurement results on the throughput.

Figure 14 

Measurement results on the expected end-to-end packet delay.

Figure 15 

Measurement results on the packet delay variance.

7. Conclusions

In the paper, we have implemented a real-time unicast module in the context of large-scale cognitive wireless networking. By opportunistically using both spectrum bandwidth and wireless nodes/radio availability, the most significant result is that the network performance can improve with larger network scale. Specifically, it is shown that the performance dynamics, that is, indicated by dataflow end-to-end delay variance, can fall to zero with higher network density. Furthermore, additional results show that dataflow throughput can be independent of the number of wireless hops or the source-to-destination distance, whereas end-to-end delay expectation and delay variance increase linearly. The obtained analytical, simulation, and experiment results support these propositions. It is also identified that node transmitting power can be an effective control knob, for the tradeoff between the real-time QoS performance metrics and resource consumptions. We suggest that the presented research could contribute to the development of diverse commercial and scientific applications with large-scale wireless networking.

Appendices

A. Calculations of the Iteration Number 𝐼

At one iteration 𝑖, let 𝑋_𝑖 denote the distance advance to the destination 𝑑, that is,𝑋_𝑖=⎧⎪⎨⎪⎩|||𝐋^𝑛𝑖−𝐋_𝑑|||−|||𝐋_𝑛𝑖+1−𝐋_𝑑|||𝐋|||,1≤𝑖<𝐼,_𝑛𝐼−𝐋_𝑑|||,𝑖=𝐼.(A.1)

Furthermore, let 𝑝_𝑋𝑖(𝑥) denote the pdf of the positive random variable 𝑋_𝑖. Given 1≤𝑖<𝐼, 𝑝_𝑋𝑖(𝑥) is decided by the distance between the node 𝑛_𝑖 and the destination 𝑑, |𝐋_𝑛𝑖−𝐋_𝑑|. Under the condition that |𝐋_𝑛𝑖−𝐋_𝑑|→∞, the probability of 𝑋_𝑖<𝑥, that is, the cumulative distribution function (cdf), is given by the probability that there is no available relay candidates to the right of the vertical line shown in Figure 16. Particularly, the relay candidates distribution is an inhomogeneous Poisson process with the density function 𝜆⋅𝜌⋅𝑓(𝑃_𝑡,𝑅,𝜁), where 𝜁 is the distance to 𝑛_𝑖. Therefore, for 𝑥≥0, the cdf of 𝑋_𝑖 is obtained:Prob𝑋_𝑖≤𝑥=𝑒^{∫−𝜆𝜌⋅∞𝑥∫∞−∞𝑓(𝑃𝑡√,𝑅,𝑢2+𝑣2)𝑑𝑢𝑑𝑣}.(A.2)The pdf of 𝑋_𝑖, 𝑝_𝑋𝑖(𝑥), can be calculated by taking the differentiation of the cdf function in (A.2), which equals to the 𝑝_Λ(𝑥) defined in (9).

Figure 16 

Calculation of 𝑝Λ(𝑥).

Therefore, 𝑝_𝑋𝑖(𝑥)→𝑝_Λ(𝑥), that is, 𝑋_𝑖 converges to Λ in distribution, when the distance |𝐋_𝑛𝑖−𝐋_𝑑|→∞. In obtaining the statistics of the iteration number 𝐼, we divide the source-to-destination distance 𝑙 into two segments, with the lengths 𝑙−√³𝑙 and √³𝑙, respectively. Let 𝐼₁ denote the first passage iteration of first segment, which is 𝐼₁=min{𝑖∣|𝐋_𝑛𝑖−𝐋_𝑑|<√³𝑙}, and let 𝐼₂ denote the rest number of iterations in the second segment. Given 𝑙, 𝐼₁ and 𝐼₂ are independent variables, and 𝐼=𝐼₁+𝐼₂. Therefore, we obtain 𝐸(𝐼)=𝐸(𝐼₁)+𝐸(𝐼₂), and 𝜎²(𝐼)=𝜎²(𝐼₁)+𝜎²(𝐼₂).

According to the above definitions, under the asymptotic condition 𝑙→∞, the set of random variables {𝑋_𝑖∣1≤𝑖≤𝐼₁} converge to the random variable Λ in distribution. Therefore, 𝐼₁ is virtually the first passage time of a positive-drift stopped random walk beyond the level 𝑙−√³𝑙, where the random walk steps are i.i.d. (The proposition is mathematically rigorous under the uniform integrability [35] of {|𝑋_𝑖|^𝑟|1≤𝑖≤𝐼₁,𝑟≥1}, which can be verified for realistic wireless propagation models 𝑓(⋅,⋅,⋅).) according to the distribution 𝑝_Λ(𝑥). Due to the results of stopped random walks [36, Theorem 9.1, page 94], there are𝐸𝐼₁=𝑙𝐸(Λ)+𝑜√𝑙,𝜎²𝐼¹=𝑙⋅𝜎²(Λ)𝐸(Λ)³+𝑜𝑙(A.3)

Now, if we assume that 𝐸(𝐼)=𝐶₁⋅𝑙^𝑐1+𝑜(𝑙^𝑐1), and 𝜎²(𝐼)=𝐶₂⋅𝑙^𝑐2+𝑜(𝑙^𝑐2), where 𝐶₁, 𝐶₂, 𝑐₁, 𝑐₂ are constants, there will be 𝐸(𝐼₂)=𝐶₁⋅𝑙^𝑐1/3+𝑜(𝑙^𝑐1/3), and 𝜎²(𝐼₂)=𝐶₂⋅𝑙^𝑐2/3+𝑜(𝑙^𝑐2/3), since the difference between 𝐼 and 𝐼₂ is only by the distances 𝑙 and √³𝑙 as defined. Due to the statistics of 𝐼₁ in (A.3), it is easy to obtain 𝐶₁=1/𝐸(Λ), 𝐶₂=𝜎²(Λ)/𝐸(Λ)³, and 𝑐₁=𝑐₂=1. Therefore, the results on the iteration number 𝐼, as given in (7) and (8), are obtained directly.

We further consider 𝐼_𝑐, which denotes the number of iterations with completed DATA transmissions, that is, the number of wireless hops. Corresponding to the random variable Λ in the previous analysis, the asymptotic step length Λ_𝑐 under the prior condition of completed DATA transmission is decided by the pdf, 𝑝_Λ𝑐(𝑥)=𝑝_Λ(𝑥)/(1−𝑒^{−𝜆𝜌Ω}), since the probability of an iteration with uncompleted DATA transmission is 𝑒^{−𝜆𝜌Ω}. Therefore, simply by replacing Λ with Λ_𝑐 in the previous analysis, the statistics of 𝐼_𝑐 can be obtained. In particular, 𝐸(𝐼_𝑐)=𝑙/𝐸(Λ_𝑐)+𝑜(√𝑙)=𝑙⋅(1−𝑒^{−𝜆𝜌Ω})/𝐸(Λ)+𝑜(√𝑙), and𝜎²(𝐼_𝑐)=𝑙⋅𝜎²(Λ_𝑐)𝐸(Λ_𝑐)³=𝑙⋅𝐸(Λ_2𝑐+𝑜(𝑙))𝐸(Λ_𝑐)³−1=𝑙⋅𝐸(Λ₂𝐸(Λ_𝑐)+𝑜(𝑙))𝐸(Λ)³⋅1−𝑒^𝜆𝜌Ω²−1−𝑒^{−𝜆𝜌Ω}=𝑙⋅[𝜎₂𝐸(Λ)+𝑜(𝑙)(Λ)𝐸(Λ)³⋅1−𝑒^{−𝜆𝜌Ω}²−𝑒^{−𝜆𝜌Ω}1−𝑒^{−𝜆𝜌Ω}𝐸(Λ)]+𝑜(𝑙).(A.4)

B. Expressions of 𝐸(Λ) and 𝜎²(Λ)

Due to the pdf 𝑝_Λ(𝑥) in (9), we obtain𝐸(Λ)=lim_𝐶→∞^𝐶0𝑥⋅𝑑𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}==lliimm_{𝐶→∞𝐶→∞}𝑥⋅𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}|^𝐶𝑥=0−^𝐶0𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑑𝑥𝐶−^𝐶0𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑑𝑥,(B.1)and similarly𝐸(Λ²)=lim_𝐶→∞^𝐶0𝑥²⋅𝑑𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}=lim_𝐶→∞𝐶²−2^𝐶0𝑥⋅𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑑𝑥.(B.2)

Therefore,𝜎²(Λ)=𝐸Λ²−𝐸Λ²=−lim_𝐶→∞2𝐶⋅^𝐶0𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑑𝑥_𝐶0𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑑𝑥²−2^𝐶0𝑥⋅𝑒^{∫−𝜆𝜌∞𝑥𝑔(𝑢)𝑑𝑢}𝑑𝑥.(B.3)

From (B.1) and (B.3), it is shown that 𝐸(Λ) increases with 𝜆, approaching a limit decided by the characteristics of 𝑔(𝑢), and 𝜎²(Λ) decreases with 𝜆, approaching zero. Since 𝑔(𝑢), as defined in (10), is a monotonically increasing function of 𝑃_𝑡, it is also shown that 𝐸(Λ) increases monotonically with 𝑃_𝑡, and 𝜎²(Λ) decreases monotonically with 𝑃_𝑡.

C. Models of the Iterative Delay and Energy Consumption

We point out that more accurate models of the iterative delay 𝜏_𝑖 (1≤𝑖<𝐼) and the iterative energy consumption 𝜖_𝑖 (1≤𝑖<𝐼) can include the distance |𝐋_𝑛𝑖−𝐋_𝑑|. Under such models, 𝜏_𝑖 and 𝜖_𝑖 will converge to the results in Sections 4.3.1 and 4.3.2, respectively, when |𝐋_𝑛𝑖−𝐋_𝑑|→∞. With similar discussion in Appendix A, this will contribute to the same end-to-end analytical results, under the asymptotic condition 𝑙→∞.

Acknowledgments

The authors would like to acknowledge the support from Natural Sciences and Engineering Research Council of Canada (NSERC) and Ontario Research Commercialization Program (ORCP).