International Journal of Digital Multimedia Broadcasting

Volume 2008, Article ID 586067, 16 pages

http://dx.doi.org/10.1155/2008/586067

## Real-Time Communications in Large-Scale Wireless Networks

Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 10 King's College Road, Toronto, ON, Canada M5S 3G4

Received 25 March 2008; Revised 13 July 2008; Accepted 17 September 2008

Academic Editor: Massimiliano Laddomada

Copyright © 2008 Liang Song and Dimitrios Hatzinakos. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

There is an emerging need for realizing real-time quality of service (QoS) over multihop wireless communications in large-scale wireless networks. The applications can include wireless mesh infrastructure for broadband Internet access supporting multimedia services, visual sensor networks for surveillance, and disaster-relief networks. However, a number of challenges still exist as revealed by recent works, where the dataflow QoS performance such as throughput and end-to-end delay can degrade fast with the number of wireless hops. We propose to use large-scale cognitive networking methods to resolve the wireless multihop challenges. By the cognitive-networking concept, data packets travel along opportunistically available paths in the network with opportunistically available spectrum in every hop. Reliable end-to-end communications can be achieved for real-time services, where we show that (1) dataflow throughput can be independent of any number of wireless hops, (2) end-to-end delay and delay variance increase linearly with the number of wireless hops, and (3) delay variance decreases to zero with higher network density. These results are supported by analysis, simulations, and experiments.

#### 1. Introduction

Large-scale wireless networks, for example, mobile ad hoc networks, wireless mesh networks, as well as wireless sensor networks, have been receiving significant attention in the past few years. Recently, there has been growing interest in realizing real-time quality of service (QoS) over multihop wireless transmissions in large-scale wireless networks. For example, the driving commercial applications include wireless mesh infrastructure for broadband Internet access [1], supporting Voice over Internet Protocols (VoIP) or Multimedia over IP, and wireless sensor network with real-time event detection/reporting, especially in video surveillance, that is, visual sensor networks [2].

Compared to the classical many-to-one (starlike) networks, such as cellular phone networks and WiFi- (IEEE 802.11) [3] enabled wireless local area networks (WLANs), a wireless mesh-based infrastructure for broadband access has the advantage of much lower capital and operational costs and can achieve higher service coverage. As one example, municipal WiFi mesh networks are rolling out in a number of major cities, providing wireless web surfing and email services to general public. Considering the popularity of Internet voice and video, further developments for supporting VoIP within the wireless mesh infrastructure are highly demanded. On the other hand, visual sensor networks appear to be one of the killer applications for wireless sensor networks, since a lot of major cities have installed public-administrated surveillance cameras for various purposes. As of now, those cameras are normally attached by cables. By adopting wireless connections, there will be significant cost reduction in the installation, which can enable much denser deployment. Moreover, related applications also include the disaster-relief wireless networks, with ad hoc formation, where the real-time information delivery can be of life and death importance.

However, many challenges still exist in realizing real-time QoS over multihop wireless networks. It has been well known that dataflow throughput, end-to-end delay, and delay jitter can degrade fast when the number of wireless hops increases [4, 5]. The limitation can be primarily introduced by dynamic network-resource availabilities including both spectrum bandwidth and wireless nodes/radios. Specifically, random spectrum availability is based on wireless fading and interferences prevailing in unlicensed bands (e.g., ISM bands); and random radio availability is due to dynamic traffic load (congestions) and other factors such as radio failure. In traditional wireless networking, the media access control (MAC) layer sets up “logic wired links” over wireless media, which assume predetermined spectrum availability. On top of it, the network layer runs ad hoc routing protocols which assume predetermined radio availability and network topology. It is difficult to have effective QoS models in the traditional network protocol stack [6], when both spectrum bandwidth and radio availabilities cannot be predetermined in large-scale wireless networks.

In order to resolve the wireless multihop limitation, there have been proposals to install multiple radios in one single wireless node [4] and design network routing protocols based on wireless link status, as well as other combined metrics [5]. Although evidence shows that improvements could be obtained, these approaches do not address real-time QoS. Moreover, existing approaches can generally suffer from the lack of scalability. Typically, the complexity of existing protocols can grow fast with the number of wireless nodes neighboring each other (higher network density), since there are a large number of links to be managed and/or tracked on every node. Other research work that investigates real-time QoS in wireless networks could be based on inappropriate assumptions for large-scale wireless networks, such as the assumption on fixed link bandwidth [7].

In this paper, we apply the methods of large-scale
cognitive networking [8]
to resolve the wireless multihop limitation. The concept of cognitive
networking opportunistically utilizes network resources including both spectrum
bandwidth and wireless node/radio availability to realize reliable
communications in large-scale wireless networks. The adopted network
architecture is the Embedded Wireless Interconnect (EWI) [8–11], where wireless linkage is
redefined introducing *abstract wireless links*. By definition, abstract
wireless links can be arbitrary functional abstractions of mutual cooperations
among proximity wireless nodes, where the categories can include broadcast,
unicast, multicast/anycast, and data aggregation, among others. Therefore, *wireless
link modules* are designed as the units for realizing different abstract
wireless links at individual wireless nodes. The architecture of EWI is then
developed upon two layers, where the bottom wireless link layer supplies a
library of wireless link modules; and the upper system layer can organize the
modules for achieving effective application programming interface (API).

Conforming to the cognitive networking concept, both the operating spectrum and the participating nodes of an abstract wireless link is opportunistically decided based on their instantaneous availabilities. By the unicast module design in the cognitive networking, data packets can take opportunistically available paths in the network from source to destination, with opportunistically available spectrum in every hop. In comparison, dynamic changes in spectrum and radio availability can create bottlenecks on the routing path of traditional wireless networking, where such bottlenecks are all resolved by the proposed methods. Therefore, reliable end-to-end communications can be achieved, by the opportunistic utilization of network resources. The presented unicast module design is also implemented by cognitive-radio prototyping [12]. In the performance analysis, real-time QoS metrics including throughput, expected end-to-end delay, and delay variance, are studied. The following propositions are obtained.

(i)All the investigated real-time QoS metrics improve with larger network scale, and the complexity on individual nodes remains constant independent of any network scale. It is also shown that the performance dynamics, as indicated by end-to-end delay variance, can diminish to zero with higher network density.(ii)Reliable communications can be achieved over a large number of wireless hops. In principle, the throughput can be independent of the source-to-destination distance; while the expected end-to-end delay, and the delay variance, increase linearly with the source-to-destination distance.(iii)Considering the resource constraints, such as energy consumption or network capacity, we identify that radio transmitting power can be an effective control knob for traffic prioritization in dealing with the tradeoff between achievable QoS performance and resource consumptions.

Therefore, the two major contributions of the paper are the following: (1) the first design & implementation of real-time communications for large-scale wireless networks; (2) the QoS performance analysis of the designed real-time unicast module. In what follows, related works are surveyed in Section 2; the unicast wireless link module design and implementation are described in Section 3; the performance analysis of real-time QoS metrics is presented in Section 4; the simulation and experiment results are reported in Section 5 and 6, respectively; the conclusion is given in Section 7.

#### 2. Related Work

Related works in cognitive networking and the EWI network architecture have previously appeared in [1, 8–12]. As a pilot architectural reference model, EWI was first introduced in the application-specific studies in wireless sensor networks [10, 11]. In [9], recent research works in the cross-layer design of wireless sensor networks are surveyed, where it is suggested that EWI can be an unified design architecture. The concept of large-scale cognitive wireless networks has been presented in [8], where EWI was adopted as the network architecture. In [1, 12], the EWI is further utilized to construct a ubiquitous wireless mesh infrastructure for broadband Internet access.

The unicast design can also be related to existing research works in opportunistic routing [13–17]. In principle, opportunistic routing schemes deal with the codesign of routing and MAC protocols. However, previous research works usually deal with single-hop performance metrics, while the contribution of this paper deals with end-to-end QoS for real-time communications. Moreover, contributions are made in large-scale cognitive networking, with a report of experiment results.

In [13], the protocol selects the next hop relay node by a slotted ACK (acknowledge) mechanism. Having successfully received a data packet, the node calculates a priority level, which is inversely proportional to the distance between the node and the packet destination. The node with the highest priority is then selected distributively as the next hop relay node. In [14, 15], a technique named GeRaF was proposed, where the node forwarding region is divided into smaller sections with different priorities, as decided by the forwarding distances. The performance analysis of GeRaF included the single-hop energy consumption and latency tradeoff, as well as the multihop count number. The study in [16] investigated the architectural aspects of opportunistic routing, where the performance of the component design is analyzed based on single-hop metrics and light-traffic scenario. In [17], the authors discussed the opportunistic relay node selection in a two-hop transmission scenario. The relay-node priority criterion is based on the gain of source-to-relay channels and relay-to-destination channels.

#### 3. Module Design and Implementation

A unicast wireless link is an abstraction of the proximity wireless nodes cooperation for unicasting. Since unicasting usually involves the information delivery from source to destination over multiple hops, the unicast module design decouples the end-to-end network behavior into proximity wireless nodes cooperation. In the following, some elements of the module design and implementation are presented. The module state-diagram is then shown.

##### 3.1. Network Address

Network address is related to the context, subject to a “cost of delivery” criteria. Let and denote the network addresses of the destination node and the source node, respectively. Given the destination of a data packet at one wireless node , a local parameter is assumed obtainable, which indicates the approximate or average cost of delivering the packet to the destination from the node independent of dynamic changes. In large-scale wireless networks, the cost of delivery is usually a function of the distance from one local node to the destination .

In location-centric networks (e.g., [10]), where wireless nodes are aware of their own locations, for example, by global position systems (GPS) or triangulation estimations, the network address of one node is solely decided by the node location coordinates . Given the destination coordinates of a packet, can be readily calculated in geometry, defined as the specific distance.

In data-centric networks (e.g., [18]), the network address can be decided by the application specific data. The cost of delivery is the application data gradient, which can be assumed as a monotonically increasing function of the distance between and .

In a data-collecting or fusion network, for example, wireless sensor networks, the sink (data collector) can broadcast a number of identity advertisement packets, which is thereon flooded in the network, by broadcasting. Every node can count the average smallest number of hops from the sink, on receiving the advertisement packets. The count number can be used as for one node . A similar approach appeared in [19] yet in a more general scenario, where the logic address, that is, a vector of the estimated distance or hop number to a group of anchors, is maintained for every wireless node. New nodes joining in the network can estimate their own logic address by acquiring the address of neighboring nodes.

Although other types of address can be applicable, we use the location address in this paper for analysis, simulations, and experiments.

##### 3.2. Radio Implementation

The terminology of cognitive radio was first suggested in [20] as an ideal-omnipotent radio which can take all the parameters into consideration for user-centric communications. Cognitive radio was later reviewed in [21, 22] as the radio with dynamic spectrum access. In [8], two propositions are further suggested for the cognitive radio for large-scale wireless networks.

(i)The radio can opportunistically sense the spectrum resource, so that the selected spectrum usage will not be interfering with other on-going wireless communications.(ii)The radio can opportunistically poll one or more other proximity radios onto the selected spectrum, so as to realize certain types of local cooperations.

The above two propositions can extend the concept of pure cognitive radio to cognitive networks, which implement both dynamic spectrum access and dynamic radio access. In the network architecture EWI, a wireless node can initiate an abstract wireless link, that is, certain types of local cooperations among a number of proximity wireless nodes. Hence, both the set of wireless nodes and the operating spectrum are opportunistically decided for an abstract wireless link. Further, the initiated link will not be interfering with other wireless communications, in accordance with the first cognitive proposition.

In the test-bed development of the paper, we prototyped an experimental tone-based cognitive radio [8, 12]. Specifically, the radio can access a group of predetermined data channels, where every data channel is also associated with two distinctive frequency tones, that is, one sensing tone and one polling tone. The sensing and polling tones are at distinctive frequency different from the data channel. The radio hardware is therefore composed of two transceivers, which are the tone transceiver and the data transceiver, respectively. On initiating an abstract wireless link, the radio senses for an available channel, with the vacant data channel and the vacant sensing/polling tones. It then broadcasts the polling tone associated to the selected channel, to poll its surrounding nodes. On detecting the rising edge of the polling tone, the surrounding nodes can decide to join in the initiated abstract wireless link based on their autonomous availability. On joining in an abstract wireless link, the radio of the surrounding nodes also broadcasts the sensing tone. As such, both sensing and polling tones protect wireless link modules from spectrum interferences. We note that the use of frequency (busy) tones has also appeared in numerous works of MAC protocols [23, 24]. However, here we have the first implementation over multiple channels, and such implementation is in the context of large-scale cognitive wireless networks.

The unicast wireless link module is programmed on the cognitive radio prototype (shown in Figure 1). The experiment hardware is a stack of two radio boards (TELOSB [25]). The radio board is composed of one IEEE 802.15.4 [26] compatible transceiver, Chipcon CC2420, one TI MSP430 Microcontroller, and the program interface. The top radio board is utilized as the tone transceiver, while the bottom radio board is utilized as the data transceiver. The two are wired up by the digital interface shown in Figure 1. Three independent data channels in the 2.4 GHz band are allotted to the radio platform, while the associated tones are also in the band of 2.4 GHz. The unicast module is programmed in the firmware of TI MSP430.

##### 3.3. Packet Relaying Process

Under an initiated wireless link, the packet relaying process picks one available relay node and sends the data information from the source or a previous relay node to the new relay node.

Four types of packets are used in the packet relaying process, which are RTS, CTS, DATA, and ACK, respectively. RTS, CTS, and ACK are control packets, while DATA is the data information unit. Let node denote the relay node at the hop , the next hop relay node is found by the following procedure, which incurs local wireless nodes cooperation in proximity, that is, a unicast wireless link.

First, the node broadcasts a RTS, including the module type (unicast), the destination address , and the self-address . Upon receiving the RTS at one node , if the condition is satisfied, the node initializes a timer, with the timeout period as . is inversely proportional to , which is determined by the specific physical radio technology, for example,where is a constant, and . is the smallest (minimal) inter frame space (delay constant), which is composed of the module processing time and the transceiver RX/TX switch time. In our current test-bed implementation, is also slotted according to the minimal carrier-sensing time. Note that the condition is enforced, so that the cost of delivery to the destination is always descending which prevents any loop.

The node then backs off and monitors the data channel for the period . If the data channel is free during that period, the node replies with a CTS, declaring itself as the next hop relay node . Otherwise, the node quits the unicast module. As such, the node with the smallest cost should be elected as the next hop relay node . After having received the CTS, the node transmits the DATA packet, and after receiving the DATA packet, the node replies an ACK, completing the round of relaying. In the case that more than one node send out their CTS simultaneously, the transmissions will collide on some specific bits, for example, the node address, where the CTS packets are different. Since this collision can be detected by certain mechanism, such as cyclic redundancy check (CRC), at , the node can use an appended procedure to differentiate one node as , either before or after the DATA transmission.

The described mechanism is illustrated in Figure 2. In the last hop, the destination is always assigned with the minimal delay, that is, .

##### 3.4. Module State Diagram

The state diagram of a single unicast module in the wireless link layer is shown in Figure 3, which can give a summary of the implementation. The definitions of the states and the associated transferring branches can be self-explainable, in accordance with previous descriptions.

The wireless link layer stays in the IDLE state, when no module is involved. It can initiate a unicast module, for example, on receiving the command from the system layer, so as to send certain amount of information (i.e., contained in one DATA packet) to a specified destination. In the IDLE state, the wireless link layer also monitors the set of polling tones, which are associated to the predetermined group of data channels. On detecting the rising edge of one polling tone, it sends one module request to the system layer. Upon approval, the wireless link layer listens to the RTS packet on the data channel, which is associated with the detected polling tone. Otherwise, it goes back to the IDLE state. Furthermore, after having received one DATA packet, the unicast module automatically relays it, if the current node is not the destination.

Table 1 shows an example set of primitive functions on
the interface between the system layer and the wireless link layer, as related
to the unicast module. The parameter *Priority* in Table 1 can be used for
specifying the traffic class, which will be explored further in Section 4.5.3.

##### 3.5. Discussion

By opportunistically dealing with network resources including both spectrum bandwidth and radio availability, data packets take opportunistically available paths with opportunistically available spectrum in every hop. Since the unicast module operates opportunistically in proximity wireless nodes, the end-to-end performance of real-time QoS, can be statistically assured, adding up the sequential proximity operations. This discussion can provide an intuitive explanation of the performance analysis results in Section 4.

Specifically, random spectrum availability is handled by the cognitive radio implementation, where a source or relay node tries to find an available data channel for initiating the unicast wireless link. Important aspects of random node/radio availability are as follows: the node deployment, mobility, and traffic congestion. In general, it indicates that the node , that is, the hop relay, would be uncertain about the availability of the next hop relay node, before the RTS probing. The RTS/CTS exchange in the module design opportunistically finds the next hop relay, in a group of available candidates.

In traditional computer networks, dynamic traffic load is limited by network congestion control. The network layer drops overflowed packets by queueing management, which is then detected by the transport layer. Such methods have been known to incur problems in wireless networks [27]. By the real-time unicast module, the paradigm of the classical “network queueing” could be transferred to “queueing in network.” In an individual relay node, a DATA packet is forwarded automatically without lengthy buffering in one local queue. Sequential DATA packets nominally take opportunistically decided paths from the source to the destination, which are buffered in the network, instead of any predetermined nodes.

Therefore, it can be intuitive to conceive that the QoS performance metrics should improve with larger network scale, since the network density contributes to the diversity (or more radio resource) that can be opportunistically exploited. This agrees with other theoretical results, for example, in [28], where it is stated that summed network transport capacity should increase with network scale, on the order of , where is the number of nodes.

Moreover, in the proposed method, any node does not need to consider or know every other possible node as a candidate for the next hop. In the formation of abstract wireless linkage, the participating nodes are decided by their autonomous availability. The unicast module design also assures that there would only be one RTS response in most cases by the carrier-sensing mechanism. Therefore, the module complexity at individual nodes remains constant, independent of the network scale.

#### 4. Performance Analysis of Real-Time QoS

The performance analysis mathematically quantifies the major propositions in the paper. The results can also provide useful network planning guidance for deployments.

##### 4.1. Objectives and Approaches

In supporting real-time QoS, the metrics of throughput, expected end-to-end delay, and delay variance, are studied for the unicast wireless link module. Two types of resource consumptions are considered, which are network capacity and network energy consumption. Therefore, we centrally explore the following three questions. (1) How do the QoS metrics change with network scale? (2) How do the QoS metrics change with the source-to-destination distance? (3) How to control the resource allocation, so as to achieve the QoS requirements?

Given the source and the destination of a unicast dataflow, let denote the source-to-destination distance. Analytical results are obtained under the asymptotic condition of long-range unicasting, , where the justification can be that long-range unicasting presents the worst-case scenario of the network performance under practical interests. On the other hand, concise analytical formulation of the QoS metrics can be obtained under this long-range asymptotic condition, which can provide analytical insights to previous questions.

##### 4.2. Analytical Models

In the performance analysis, models about node distribution, wireless channel, power consumption, bandwidth availability, and the IDLE probability are adopted.

###### 4.2.1. Node Distribution Model

The node distribution is modeled as a two-dimensional (2D) Poisson process [29], with the node density . That is, given an area of the size in the field, the number of nodes in the area follows Poisson distribution with the parameter . The Poisson modeling can be typical for random node distribution and/or random mobility.

###### 4.2.2. Wireless Channel Model

Given an arbitrary transmitting node , and a receiving node , the successful transmission probability, in general, is a function of the transmitting power , the radio data rate , and the distance . We denote this function as .

For example, if small-scale Rayleigh fading [30] is assumed, the channel model can be given byIn (2), is a propagation-loss constant; is the receiver-noise power spectrum density; is the path loss component in wireless channel [30]; is a threshold constant representing the receiver sensitivity; and is a unit-mean exponential random variable. This model will also be utilized to obtain the numerical results in Section 5.

###### 4.2.3. Power Consumption Model

Referring to the state diagram in Figure 3, the node power consumption at the IDLE state is denoted by , that is, the low-power monitoring of the polling tones; the power consumption at the states of “Send DATA”, “Send RTS”, “Send CTS”, and “Send ACK”, is modeled bywhere is the transmitter circuits power consumption; is a constant decided by the RF power amplifier efficiency [31]; and is the transmitting power as defined perviously. The node power consumption in other states or time is modeled by a constant , which denotes the receiving (or idle listening) power consumption.

###### 4.2.4. Bandwidth Availability and the IDLE Probability

When initiating a unicast module, we assume that the node can always find an available channel by the cognitive radio without significant delay. The IDLE probability is the probability that a given node is in the IDLE state (shown in Figure 3), and can be engaged in a unicast wireless link being initiated by other nodes around it. We assume that there is a fixed (or lower bounded) IDLE probability “”, in the analysis.

With limited network capacity and arbitrary traffic load, these two assumptions need to be ensured by an appropriate call admission control (CAC) mechanism. On the other hand, given analogous experiences of the popular video streaming on peer-to-peer overlay networks, it could also be appropriate to assume no CAC mechanism, while the traffic volume demanding can always fall below the planned (or unplanned) network capacity. In large-scale wireless networks, the capacity could be of relatively cheap resources, given abundant wireless nodes and the use of unlicensed band, while the traffic load is not necessarily related to the number of wireless nodes. These considerations give the optimization formulations under different types of resource consumptions in Section 4.5.

###### 4.2.5. Other Parameters and Notations

The packet lengths (bit) of RTS, CTS, DATA, and ACK are denoted by , , , and , respectively. denotes the time delay in the “channel sensing” state in Figure 3, and denotes the time delay in the “Backoff ” state. We also use the notations and to denote the mean and the variance of a random variable.

##### 4.3. Iterative Performance Analysis

“Iterative performance” can be similar to “single-hop performance.” However, with the notation of “iterative performance,” we also consider the scenario where the next hop relay node is not found in the transferring branch from “Send RTS” to “channel sensing” of Figure 3. The reason resides in the random node availability, for example, no potential relay candidate is in the IDLE states. Due to this difference, the terminology “iteration” instead of “hop” is used in the following analysis, and “iterative delay” indicates the time delay of an iteration, whereas “iterative energy consumption” indicates the network energy consumption in one iteration.

Particularly, consider that there are totally iterations from the source to the destination . Let denote the revelent set of nodes, which initiates the unicast wireless link. Obviously, there is , and sends the DATA packet directly to the destination . The iteration number is therefore a random variable.

###### 4.3.1. Iterative Delay

Let denote the iterative delay at an iteration . (Other possible models are also discussed in Appendix C, which contribute to the same end-to-end performance analysis results.) If the DATA transmission is completed in the iteration, can be calculated by the formula . Otherwise, if the DATA transmission is uncompleted since no relay is found, can be given by . Obviously, for the last iteration (), the DATA transmission is completed.

When , the probability of uncompleted DATA transmission depends on the relay candidates availability on the forwarding plane. Due to the Poisson node distribution model with the density , and the node IDLE probability , the distribution of relay candidates conforms to an inhomogeneous Poisson process with the density function on the forwarding plane, where indicates the distance to the node . Therefore, the “uncompleted” probability is decided by the “equal-zero” probability of a Poisson random variable with the mean , which is given by . is always positive, and is given by the integration of the channel model on the forwarding plane:

According to the above analysis, can be modeled as a set of random variables with identical distributions, where the expectation is

###### 4.3.2. Iterative Energy Consumption

Let denote the iterative energy consumption at an iteration . (Other possible models are also discussed in Appendix C, which contribute to the same end-to-end performance analysis results.) If the DATA transmission is not completed in the iteration, the energy consumption of the node is , according to the described models. The energy consumption of one relay candidate is given by . The number of the relay candidates around , which receive the RTS, is a Poisson random variable, with the mean value . Here can be still given by (4), but is integrated on the backward plane, that is, , instead of the forwarding plane. Therefore, the expected iterative energy consumption under uncompleted DATA transmission is the summation: .

If the DATA transmission is completed in the iteration, the energy consumption of the node is . The energy consumption of one relay candidate, which is not elected as the next node , is still . The number of the relay candidates is a Poisson random variable with the mean value , including both the forward and the backward planes. On the node , the additional energy consumption is . Therefore, the expected iterative network energy consumption under completed DATA transmission is .

Since the DATA transmission is always completed in the last iteration, is directly obtained from the previous discussion. When , similar to the previous analysis of , the probability of uncompleted DATA transmission is decided by . Therefore, can be modeled as a set of random variables with identical distributions, where the expectation is

###### 4.3.3. Number of Iterations

Given all the parameters, the total iteration number is a random variable decided by the source-to-destination distance . It is shown in Appendix A that the mean value and the variance arerespectively.

In the above, the random variable generally indicates the forwarding distance of one single iteration. is defined with the pdf (probability density function) :where

Proved in Appendix B, we point out that the expectation is an increasing function of the node density , but is upper bounded, as decided by the channel model . Furthermore, the variance decreases to zero with higher . Given (7) and (8), both and can decrease with higher node density . These relations can also be intuitively understood, which will be exploited for interpreting the end-to-end performance analysis results next.

##### 4.4. End-to-End Performance Analysis

Analytical results are presented for the maximal throughput , the expected end-to-end delay , the end-to-end delay variance , and the expected network energy consumption per DATA packet , of a dataflow from the source to the destination , under the long-range asymptotic condition . The discussion shows how these metrics change with the source-to-destination distance , as well as the network density .

###### 4.4.1. Throughput

The maximal throughput is decided by how many DATA packets can be sent out from the source to the destination in a period of time, subject to the condition that this time period is much larger than the packet traveling time. According to previous analysis in Section 4.3.1, the probability of the completed DATA transmission in an iteration is . Therefore, the expected number of iterations initiated by for one DATA packet is , and the associated time expectation is .

Given the DATA packet length , the maximal throughput is thus obtained as

It is shown that the maximal throughput can be independent of the source-to-destination distance . It also increases monotonically with the node density , approaching the limit decided by the radio data rate and the overhead ratio.

###### 4.4.2. Expected End-to-End Delay

The end-to-end delay is given by . The expected end-to-end delay iswhere (a) is given by the definition; (b) is due to the fact that are of identical probability distribution [32]; and (c) is obtained directly from (5) and (7).

Therefore, the expected end-to-end delay increases linearly with the source-to-destination distance . It also decreases with the node density , approaching a bounded limit.

###### 4.4.3. Delay Variance

Let denote the number of the iterations with completed DATA transmission, that is, the number of wireless hops, and let denote the number of the iterations with uncompleted DATA transmission. and are independent random variables, and . The delay variance is calculated by the following:where (a) is given by the definition; (b) is due to the analysis in Section 4.3.1, that is, the iterative delay under completed and uncompleted transmissions respectively; (c) is due to the fact that ; (d) is obtained from the result of in (8), as well as the derived in (A.4) of Appendix A.

Therefore, the end-to-end delay variance increases linearly with the source-to-destination distance . also diminishes to zero with the network density , since the component also decreases to zero with higher .

###### 4.4.4. Network Energy Consumption

The expected network energy consumption for one DATA packet iswhere (a) and (b) are given by the definition and the fact that are of identical probability distributions [32]; (c) is obtained directly from (7), and the component can be replaced by the result in (6). Therefore, also increases linearly with the source-to-destination distance .

##### 4.5. Optimizations under QoS Constraints

Consider that the dataflow has certain QoS requirements on throughput, expected end-to-end delay, and delay variance, for example, , , and . Here, , , and are the threshold constants representing those QoS constraints, respectively. The optimizations aim to minimize the resource consumption, subject to these QoS constraints.

Two types of resource consumptions, including network energy consumption and network capacity, are considered here, which lead to different optimization formulations. As pointed out in Section 4.2.4, these two formulations can be representing different networking scenarios. In capacity limited networks, every node can have cabled power supply. The number of nodes is therefore limited by engineering costs, and the possible use of long-range radio also limits the available bandwidth. On the other hand, in energy limited networks, the nodes can be powered by energy scavenging, for example, solar cells. Therefore, the number of wireless nodes can be large, which can also use short-range radio with vast unlicensed bandwidth. These contribute to large network capacity. Other works with similar considerations have been discussed in [33].

###### 4.5.1. Capacity Limited Networks

The objective of the optimization is to minimize the transmitting power , so as to limit the occupied geographic area of an unicast link. In accordance with the results in [28], this approach can save the use of wireless network capacity. Therefore, the optimization formulation is

###### 4.5.2. Energy Limited Networks

The objective of the optimization is to minimize the expected network energy consumption per DATA packet :

###### 4.5.3. Solutions and Traffic Priority

In order to solve the optimizations in (15) and (16), the transmitting power can be an ideal control knob (optimization variable), for the resource allocation. Both (15) and (16) can be found as convex optimization problems over , where efficient solutions are guaranteed [34].

Specifically, as shown in (11), monotonically increases with , since (defined in (4)) monotonically increases with , provided by any realistic channel model . As shown in (12), monotonically decreases with , since monotonically increases with (proved in Appendix B). also monotonically decreases with in (13), since monotonically decreases with (proved in Appendix B). Moreover, as shown in (14), can be verified as a convex function of , where both and monotonically increase with . Due to the described monotonicity, the convexity of the optimizations in (15) and (16) can then be obtained, over the control knob .

Therefore, we suggest that the transmitting power can provide an effective control knob for the tradeoff between the resource consumptions and the real-time QoS requirements. In fact, different levels of can be configured for different classes of traffics, for example, voice, video (streaming), or data, which then decide the unicasting “priority”.

Further investigation may also include the radio data rate , so as to generate the problem of joint power and rate control. Unfortunately, the convexity over cannot be obtained for a general channel propagation model . Future studies may address the problem on some specific models.

##### 4.6. Summary

In summary, the analytical results indicate that all the investigated real-time QoS metrics can improve with larger network scale. In particular, the maximal throughput increases with the network density to a predetermined bound, while the expected end-to-end delay decreases with , approaching a predetermined bound, as decided by other related parameters. On the other hand, the delay variance falls to zero with higher network density , which indicates that the network performance can be made arbitrarily stable, simply by dropping more nodes in the network. For supporting long-range communications over a relatively large number of wireless hops, the analytical results show that the maximal throughput can be independent of the source-to-destination distance, while both the expected delay and the delay variance increase linearly with the source-to-destination distance. Furthermore, it is identified that the radio transmitting power can be an ideal control knob for the tradeoff between the real-time QoS requirements and the resource consumptions, for example, energy consumption or network capacity.

We consider that all these propositions could be intuitively conceivable for large-scale wireless networks. Since the analytical results are obtained under the long-range asymptotic condition , they may present the worst-case scenario analysis for practical considerations. In the next section, the simulation results are also provided, showing how these metrics converge.

#### 5. Simulation Results

Some fixed parameters in the simulations are listed in Table 2. In particular, the radio transmission parameters are conforming to the developed test-bed, while others, for example, power consumption and channel model parameters, are typical for indoor short-range radios. For example, under these parameters, the radio range is about meters, when the transmitting power is .

Given the source and the destination nodes, separated by the distance , random Poisson points are generated, for every iteration. The generation of the Poisson points conforms to the density on the 2D plane, which represent the available wireless nodes. DATA packets are then “sent out” consecutively from the source to the destination, according to the described real-time unicast module. The end-to-end delay instance of every DATA packet is then recorded, from which the simulated end-to-end delay expectation and the simulated delay variance are estimated by standard statistical methods. The simulated throughput is also obtained, by dividing the data bulk size with the time consumption of sending out the DATA packets at the source. In the following figures, we also compare the simulation results to the analytical results side-by-side, where the lower-order terms, for example, or , are neglected in plotting the analytical results.

Figures 4, 5, and 6 show how the QoS metrics change with the source-to-destination distance , where the node density is fixed at . Then Figures 7, 8, and 9 show how the QoS metrics change with the network density , where the source-to-destination is fixed at . The number of wireless hops in the simulations can be up to 20. All the obtained simulation results match well with the analytical results, as well as the main propositions given in Section 4.6. Especially, Figures 4, 5, and 6 show that close-matching also exists in short-range unicasting, although the analytical results were obtained under long-range asymptotic approximation.

Moreover, Figure 10 shows how the expected network energy consumption per DATA packet changes with the transmitting power , which is observed as a convex function. It is also observed in Figure 10 that the expected network energy consumption increases linearly with , and it decreases with in the simulation. These simulation results conform to previous performance analysis.

#### 6. Experiment Results

In the test-bed (Figure 11), the source and the destination nodes are connected to a laptop computer for the purpose of collecting experiment records. The wireless nodes are based on the cognitive radio prototype (Figure 1), with a stack of two radio boards. The cognitive radio can opportunistically access data channels in 2.4 GHz band, as described in Section 3.2, where each channel has an air-interface data rate as defined in IEEE 802.15.4 [26].

The real-time unicast module is implemented in the cognitive radio prototype, with preconfigured address. The packet length parameters, that is, , , , , are about the same as those listed in Table 2. The node transmitting power is however fixed at . The experiment results on throughput, end-to-end delay expectation, and delay variance, are obtained by the following methods. (1) Throughput is calculated at the source, where the laptop computer forwards DATA packets to the source node. Since the source node accepts a new DATA packet from the computer only after the previous one has been sent out, the computer can count the total time expenditure in sending the DATA packets. The measured throughput is obtained by dividing the bulk size with the time expenditure. (2) On receiving a DATA packet, the destination also forwards the DATA packet to the laptop computer, where the end-to-end delay instance of every DATA packet is calculated at the computer by the difference between the packet departure and arrival time. The measured end-to-end delay expectation and the delay variance are then obtained by standard statistical methods. Since the operating system running at the laptop computer (Windows XP Professional Edition) is by no means real-time, we tried to minimize the computer running threads in the experiments.

The experiments were conducted in a standard office environment, where external interferences in the utilized 2.4 GHz band can be intensive, for example, from adjacent WLAN hot spots. Two sets of experiments are illustrated in Figure 12. In Set-I, the nodes are placed in a straight line, such that the four configurations correspond approximately to one-hop to four-hop communications. For example, in the second configuration of Set-I, the node placement is such that the chance of direct transmission from to is small. In Set-II, more relay nodes are added to every hop being illustrated in Figure 12. The purpose of the two-set experiments is to compare how the designed unicast module can improve the test-bed QoS performance. In fact, Set-I experiments can be comparable to traditional wireless networking where a routing path is predetermined with best available spectrum in every hop. However, in Set-II experiments, data is forwarded from to opportunistically without any predetermined path.

Figures 13, 14, and 15 show the measurement results of the two-set experiments on throughput, expected end-to-end delay, and delay variance, respectively. Evidently, the results of Set-II are strictly better than those of Set-I, beyond the first configuration. In Figure 13, the throughput of Set-I is reduced by more than from the first to the second configuration, that is, from one-hop to two-hop, due to the impact of half-duplex radios. A comparable reduction of throughput is not observed in Set-II. In general, the throughput of Set-II changes smoothly with different configurations (number of wireless hops), where the limited reduction can be due to the limited number of nodes. In Figures 14 and 15, it is further observed that the expected end-to-end delay and the delay variance increase linearly with the number of hops (configuration index) in Set-II experiments; whereas they tend to increase much faster in Set-I experiments.

#### 7. Conclusions

In the paper, we have implemented a real-time unicast module in the context of large-scale cognitive wireless networking. By opportunistically using both spectrum bandwidth and wireless nodes/radio availability, the most significant result is that the network performance can improve with larger network scale. Specifically, it is shown that the performance dynamics, that is, indicated by dataflow end-to-end delay variance, can fall to zero with higher network density. Furthermore, additional results show that dataflow throughput can be independent of the number of wireless hops or the source-to-destination distance, whereas end-to-end delay expectation and delay variance increase linearly. The obtained analytical, simulation, and experiment results support these propositions. It is also identified that node transmitting power can be an effective control knob, for the tradeoff between the real-time QoS performance metrics and resource consumptions. We suggest that the presented research could contribute to the development of diverse commercial and scientific applications with large-scale wireless networking.

#### Appendices

#### A. Calculations of the Iteration Number

At one iteration , let denote the distance advance to the destination , that is,

Furthermore, let denote the pdf of the positive random variable . Given , is decided by the distance between the node and the destination , . Under the condition that , the probability of , that is, the cumulative distribution function (cdf), is given by the probability that there is no available relay candidates to the right of the vertical line shown in Figure 16. Particularly, the relay candidates distribution is an inhomogeneous Poisson process with the density function , where is the distance to . Therefore, for , the cdf of is obtained:The pdf of , , can be calculated by taking the differentiation of the cdf function in (A.2), which equals to the defined in (9).

Therefore, , that is, converges to in distribution, when the distance . In obtaining the statistics of the iteration number , we divide the source-to-destination distance into two segments, with the lengths and , respectively. Let denote the first passage iteration of first segment, which is , and let denote the rest number of iterations in the second segment. Given , and are independent variables, and . Therefore, we obtain , and .

According to the above definitions, under the asymptotic condition , the set of random variables converge to the random variable in distribution. Therefore, is virtually the first passage time of a positive-drift stopped random walk beyond the level , where the random walk steps are i.i.d. (The proposition is mathematically rigorous under the uniform integrability [35] of , which can be verified for realistic wireless propagation models .) according to the distribution . Due to the results of stopped random walks [36, Theorem 9.1, page 94], there are

Now, if we assume that , and , where , , , are constants, there will be , and , since the difference between and is only by the distances and as defined. Due to the statistics of in (A.3), it is easy to obtain , , and . Therefore, the results on the iteration number , as given in (7) and (8), are obtained directly.

We further consider , which denotes the number of iterations with completed DATA transmissions, that is, the number of wireless hops. Corresponding to the random variable in the previous analysis, the asymptotic step length under the prior condition of completed DATA transmission is decided by the pdf, , since the probability of an iteration with uncompleted DATA transmission is . Therefore, simply by replacing with in the previous analysis, the statistics of can be obtained. In particular, , and

#### B. Expressions of and

Due to the pdf in (9), we obtainand similarly

Therefore,

From (B.1) and (B.3), it is shown that increases with , approaching a limit decided by the characteristics of , and decreases with , approaching zero. Since , as defined in (10), is a monotonically increasing function of , it is also shown that increases monotonically with , and decreases monotonically with .

#### C. Models of the Iterative Delay and Energy Consumption

We point out that more accurate models of the iterative delay () and the iterative energy consumption () can include the distance . Under such models, and will converge to the results in Sections 4.3.1 and 4.3.2, respectively, when . With similar discussion in Appendix A, this will contribute to the same end-to-end analytical results, under the asymptotic condition .

#### Acknowledgments

The authors would like to acknowledge the support from Natural Sciences and Engineering Research Council of Canada (NSERC) and Ontario Research Commercialization Program (ORCP).

#### References

- L. Song, “Wireless mesh infrastructure supporting broadband internet with multimedia services,” in
*Proceedings of the 4th IEEE International Conference on Circuits and Systems for Communications (ICCSC '08)*, pp. 792–796, Shanghai, China, May 2008. View at Publisher · View at Google Scholar - D. Kundur, C.-Y. Lin, and C.-S. Lu, “Visual sensor networks,”
*EURASIP Journal on Advances in Signal Processing*, vol. 2007, Article ID 21515, 3 pages, 2007. View at Publisher · View at Google Scholar - ANSI/IEEE Standard for Local and Metropolitan Area Networks Part 11 (802.11), “Wireless LAN medium access control (MAC) and physical layer (PHY) specifications,” revised 2003.
- R. Draves, J. Padhye, and B. Zill, “Routing in multi-radio, multi-hop wireless mesh networks,” in
*Proceedings of the 10th Annual International Conference on Mobile Computing and Networking (MOBICOM '04)*, pp. 114–128, Philadelphia, Pa, USA, September 2004. View at Publisher · View at Google Scholar - Y. Zhu, H. Liu, M. Wu, D. Li, and S. Mathur, “Implementation experience of a prototype for video streaming over wireless mesh networks,” in
*Proceedings of the 4th Annual IEEE Consumer Communications and Networking Conference (CCNC '07)*, pp. 23–28, Las Vegas, Nev, USA, January 2007. View at Publisher · View at Google Scholar - A. Kumar, D. Manjunath, and J. Kuri,
*Communication Networking: An Analytical Approach*, Morgan Kaufmann, San Francisco, Calif, USA, 2004. - P. Djukic and S. Valaee, “Link scheduling for minimum delay in spatial re-use TDMA,” in
*Proceedings of the 26th IEEE International Conference on Computer Communications (INFOCOM '07)*, pp. 28–36, Anchorage, Alaska, USA, May 2007. View at Publisher · View at Google Scholar - L. Song, “Cognitive networks: standardizing the large scale wireless systems,” in
*Proceedings of the 5th IEEE Consumer Communications and Networking Conference (CCNC '08)*, pp. 988–992, Las Vegas, Nev, USA, January 2008. View at Publisher · View at Google Scholar - L. Song and D. Hatzinakos, “Embedded wireless interconnect for sensor networks: concept and example,” in
*Proceedings of the 4th Annual IEEE Consumer Communications and Networking Conference (CCNC '07)*, pp. 850–854, Las Vegas, Nev, USA, January 2007. View at Publisher · View at Google Scholar - L. Song and D. Hatzinakos, “A cross-layer architecture of wireless sensor networks for target tracking,”
*IEEE/ACM Transactions on Networking*, vol. 15, no. 1, pp. 145–158, 2007. View at Publisher · View at Google Scholar - L. Song and D. Hatzinakos, “Architecture of wireless sensor networks with mobile sinks: sparsely deployed sensors,”
*IEEE Transactions on Vehicular Technology*, vol. 56, no. 4, pp. 1826–1836, 2007. View at Publisher · View at Google Scholar - L. Song, D. Hatzinakos, and X. Wang, “Wireless mesh infrastructure for ubiquitous voice and video,” in
*Proceedings of the 5th IEEE Consumer Communications and Networking Conference (CCNC '08)*, pp. 1267–1268, Las Vegas, Nev, USA, January 2008. View at Publisher · View at Google Scholar - S. Biswas and R. Morris, “Opportunistic routing in multi hop wireless networks,” in
*Proceedings of the 2nd ACM Workshop on Hot Topics in Networks (HotNets '03)*, Cambridge, Mass, USA, November 2003. - M. Zorzi and R. R. Rao, “Geographic random forwarding (GeRaF) for ad hoc and sensor networks: energy and latency performance,”
*IEEE Transactions on Mobile Computing*, vol. 2, no. 4, pp. 349–365, 2003. View at Publisher · View at Google Scholar - M. Zorzi and R. R. Rao, “Geographic random forwarding (GeRaF) for ad hoc and sensor networks: multihop performance,”
*IEEE Transactions on Mobile Computing*, vol. 2, no. 4, pp. 337–348, 2003. View at Publisher · View at Google Scholar - R. C. Shah, S. Wietholter, A. Wölisz, and J. M. Rabaey, “Modeling and analysis of opportunistic routing in low traffic scenarios,” in
*Proceedings of the 3rd International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WIOPT '05)*, pp. 294–304, Trentino, Italy, April 2005. View at Publisher · View at Google Scholar - A. Bletsas, A. Khisti, D. P. Reed, and A. Lippman, “A simple cooperative diversity method based on network path selection,”
*IEEE Journal on Selected Areas in Communications*, vol. 24, no. 3, pp. 659–672, 2006. View at Publisher · View at Google Scholar - C. Intanagonwiwat, R. Govindan, D. Estrin, J. Heidemann, and F. Silva, “Directed diffusion for wireless sensor networking,”
*IEEE/ACM Transactions on Networking*, vol. 11, no. 1, pp. 2–16, 2003. View at Publisher · View at Google Scholar - Q. Cao and T. Abdelzaher, “Scalable logical coordinates framework for routing in wireless sensor networks,”
*ACM Transactions on Sensor Networks*, vol. 2, no. 4, pp. 557–593, 2006. View at Publisher · View at Google Scholar - J. Mitola III and G. Q. Maguire Jr., “Cognitive radio: making software radios more personal,”
*IEEE Personal Communications*, vol. 6, no. 4, pp. 13–18, 1999. View at Publisher · View at Google Scholar - S. Haykin, “Cognitive radio: brain-empowered wireless communications,”
*IEEE Journal on Selected Areas in Communications*, vol. 23, no. 2, pp. 201–220, 2005. View at Publisher · View at Google Scholar - I. F. Akyildiz, W.-Y. Lee, M. C. Vuran, and S. Mohanty, “NeXt generation/dynamic spectrum access/cognitive radio wireless networks: a survey,”
*Computer Networks*, vol. 50, no. 13, pp. 2127–2159, 2006. View at Publisher · View at Google Scholar - Z. J. Haas and J. Deng, “Dual busy tone multiple access (DBTMA)-a multiple access control scheme for ad hoc networks,”
*IEEE Transactions on Communications*, vol. 50, no. 6, pp. 975–985, 2002. View at Publisher · View at Google Scholar - F. A. Tobagi and L. Kleinrock, “Packet switching in radio channels: part II. The hidden terminal problem in carrier sense multiple-access
and the busy-tone solution,”
*IEEE Transactions on Communications*, vol. 23, no. 12, pp. 1417–1433, 1975. View at Google Scholar - J. Polastre, R. Szewczyk, and D. Culler, “Telos: enabling ultra-low power wireless research,” in
*Proceedings of the 4th International Symposium on Information Processing in Sensor Networks (IPSN '05)*, pp. 364–369, Los Angeles, Calif, USA, April 2005. View at Publisher · View at Google Scholar - IEEE Standard for Local and Metropolitan Area Networks Part 15.4 (802.15.4), “Wireless medium access control (MAC) and physical (PHY) layer specifications for low-rate wireless personal area networks (LR-WPANs),” 2003.
- R. Abdelmoumen, M. Malli, and C. Barakat, “Analysis of TCP latency over wireless links supporting FEC/ARQ-SR for error recovery,” in
*Proceedings of IEEE International Conference on Communications (ICC '04)*, vol. 7, pp. 3994–3998, Paris, France, June 2004. - P. Gupta and P. R. Kumar, “Towards an information theory of large networks: an achievable rate region,”
*IEEE Transactions on Information Theory*, vol. 49, no. 8, pp. 1877–1894, 2003. View at Publisher · View at Google Scholar - J. F. C. Kingman,
*Poisson Processes*, Clarendon Press, Oxford, UK, 1993. - T. S. Rappaport,
*Wireless Communications: Principles and Practice*, Prentice Hall, Upper Saddle River, NJ, USA, 1996. - B. Otis, Y. H. Chee, and J. Rabaey, “A 400 $\mu $W Rx, 1.6 mW Tx super-regenerative transceiver for wireless sensor networks,” in
*Proceedings of the 42nd IEEE International Solid-State Circuits Conference (ISSCC '05)*, vol. 1, pp. 396–606, San Francisco, Calif, USA, February 2005. View at Publisher · View at Google Scholar - S. M. Ross,
*Stochastic Processes*, John Wiley & Sons, New York, NY, USA, 1983. - S. Verdu, “Spectral efficiency in the wideband regime,”
*IEEE Transactions on Information Theory*, vol. 48, no. 6, pp. 1319–1343, 2002. View at Publisher · View at Google Scholar - S. Boyd and L. Vandenberghe,
*Convex Optimization*, Cambridge University Press, Cambridge, UK, 2004. - P. Billingsley,
*Convergence of Probability Measures*, John Wiley & Sons, New York, NY, USA, 1968. - A. Gut,
*Stopped Random Walks: Limit Theorems and Applications*, Springer, New York, NY, USA, 1988.