Abstract
Augmented reality (AR) on a head-mounted display is conveniently supported by a wearable wireless network. If, in addition, the AR display is moderated to take account of the cognitive load of the wearer, then additional biosensors form part of the network. In this paper, the impact of these additional traffic sources is assessed. Rateless coding is proposed to not only protect the fragile encoded video stream from wireless noise and interference but also to reduce coding overhead. The paper proposes a block-based form of rateless channel coding in which the unit of coding is a block within a packet. The contribution of this paper is that it minimizes energy consumption by reducing the overhead from forward error correction (FEC), while error correction properties are conserved. Compared to simple packet-based rateless coding, with this form of block-based coding, data loss is reduced and energy efficiency is improved. Cross-layer organization of piggy-backed response blocks must take place in response to feedback, as detailed in the paper. Compared also to variants of its default FEC scheme, results from a Bluetooth (IEEE 802.15.1) wireless network show a consistent improvement in energy consumption, packet arrival latency, and video quality at the AR display.
1. Introduction
Augmented reality (AR) allows a video display of the outside world to be supplemented with computer-generated graphics, annotations, instrument readings, and other sources of information [1]. Emergency workers typically view a head-mounted display (HMD), which may be partially mediated, that is, the subject is also able to view the outside world directly. (A Carl Zeiss look-around HMD is suitable for this task. It supports true color with maximum resolution of pixels, though currently fed through a US B cable, refer to http://www.wearitatwork.com/Carl-Zeiss-Look-around-HMD.173.0.html.) Unfortunately, the level of information delivered to the operative may cause saturation (cognitive overload) due to the limited capacity of human memories [2, 3]. Cognitive overload may cause an operative’s performance to deteriorate. For example, firefighters as they pass through the rooms may have a steerable view of a scene in a burning building with the addition of a room plan within the building and possibly a feed from wireless sensor nodes in the building when the view is impaired [4]. Another example of how cognitive load can impact upon an AR includes the displays presented to some aircraft crew and military personnel [5]. To remedy this problem, in augmented cognition [6], biosensors upon the person feedback information to the AR unit and these act to reduce the level of viewable information. In augmented group cognition, that information may come from other personnel such as other firefighters within a building but again the cognitive load arising from the information transfer is controlled by inputs from biosensors. However, if feedback is over a wireless network then the extra traffic arising from bisosensor data in addition to the AR video stream will present difficulties.
Wearable wireless networks that include biosenors are generally composed of low-power devices, and consequently energy efficiency is an important issue. The devices are also close to each other. This proximity of wireless devices implies two things: firstly, that the signal propagation time is negligible with the result that immediate feedback from the receiver is possible, and secondly, the chance of data losses owing to wireless channel interferences is higher. More specifically, there can be local scattering due to changes in the geometry of the wearer’s body. Unfortunately, the video stream that supports AR is highly susceptible to errors as it must be compressed to avoid the unacceptably large datarates of raw video. Because successive video frames are broadly similar (except at scene cuts) to gain encoding efficiency only the difference between successive frames is coded. Removing temporal redundancy introduces a dependency on previously transmitted data, which implies that lost packets will have an impact on future frames. Consequently, loss of packets from an intra- or spatially-coded I-frame is particularly harmful, though loss of data from other predictively-coded frames (ones employing motion compensation to reduce temporal redundancy) also has a knock-on effect at the decoder. Video frames are generally organized as a group of pictures (GOPs) of typically 12 or 15 pictures or frames initiated by an I-frame, but until the arrival of the next I-frame a decoder will face problems in reconstructing the sequence if reference frame packets are lost. To reduce latency redundant data in the form of forward error control (FEC) is introduced rather than rely on error control through some form of automatic repeat request (ARQ) for retransmissions of complete packets. However, introducing FEC increases transmission overhead and in general further stretches the capacity of a wearable wireless network, already under strain because of the additional biosensor traffic.
Our proposed solution is a power efficient algorithm based on a novel block-based rateless FEC that tries to avoid unnecessary packet retransmissions. When a packet arrives at a receiver, it is declared lost because it cannot be decoded, though individual blocks within it may become decodable if redundant blocks were to be available. If a packet or rather the blocks within it is found to be not decodable, the method resends redundant blocks in a current packet rather than resend the complete packet. Immediate feedback from the receiver, which in some wireless technologies such as Bluetooth comes for free because of its time division duplex (TDD) operation, allows the implementation of the block-based rateless scheme. The focus of this paper is the AR application with biosensors in which Bluetooth is modeled as a suitable wireless technology. However, the block-based rateless coding described in the paper is applicable to centrally scheduled, packet-switched wireless networks with low latency feedback. IEEE 802.11e operating with hybrid coordinator function (HCF) controlled channel access (HCCA) can operate in this fashion and could be considered for a wearable wireless network, as this variant of IEEE 802.11 was designed with multimedia traffic in mind. Though obviously unsuitable for this AR application, the IEEE 802.16 is also centrally scheduled and uses TDD.
It is the datarate levels that present the most critical impact upon the coexistent video stream needed to support AR rather than the traffic patterns generated by the biosensors. This is because the need to support video of a suitable quality already stresses a wireless network. We have tested video transmission against these critical data rates and show that, with our form of block-based rateless coding, overhead is reduced, while error protection is preserved. Therefore, the main contribution of the paper is the block-based rateless coding which reduces overhead and consequently makes running the AR application with additional biosensor traffic easier to accomplish. As part of our introduction, we now further describe the rateless coding solution and the main features of the wearable wireless network. We also consider the suitability of existing wireless technologies for this application.
1.1. Block-Based Rateless Coding
Because wireless networks are subject to various forms of interference such as fast and slow fading, shadowing, and radio frequency noise, the paper proposes that Fountain or rateless channel error coding [7] of the video stream will reduce erasures, while jointly improving energy consumption taken up in transmission of the video stream. Error bursts arising from slow fading are difficult to correct with standard block-based channel coding but can be addressed through erasure coding. This paper demonstrates an innovatory form of block-based rateless coding that is better able to reduce transmission energy consumption through reduced overhead than other FEC-based methods, including packet-based rateless coding. As nonrateless block-based FEC is a default option in the wireless network under test, by comparison with the default scheme the reader will be able to judge the relative advantage of opting for block-based rateless coding.
With the proposed block-based method of rateless coding, redundancy is reduced because the unit of coding is not a packet but a block within a packet. By piggybacking redundant blocks onto newly transmitted packets, redundancy is incrementally achieved until either prior video-bearing packets with erased blocks are reconstructed or the display deadline of the frame of which that packet is a part expires. Furthermore, compared to the other FEC methods tested, fewer packets are dropped through late arrival, which is important for video in general and for a real-time application in particular. Lower average delay leads to smaller playout buffers, with consequent saving in memory energy consumption. Compared to simple packet-based rateless coding, data loss is reduced and energy efficiency is improved in worsening channel conditions. However, block-based rateless coding requires cross-layer attention to be paid to the feedback channel and block packing for multiple packets must be recursively applied in the reply packet.
In the multimedia broadcast multicast system (MBMS) [8, 9], rateless coding at the application layer has been introduced by 3GPP for video streaming. However, MBMS differs from the use of rateless coding in this paper because (a) it is for multicast not for unicast, (b) it sends separate FEC packets and, hence, is properly described as packet-based, and (c) there is no feedback, because rateless coding is employed for its maximum distance separable property, that is, the source packets can be reconstructed with high probability from any set of k or just slightly more than k received symbols. In [10], rateless coding is applied to unicast streaming over the fixed Internet but at the packet level and with large values of k.
1.2. Wearable Wireless Network
We model a wearable wireless network to find the impact of data traffic from the biosensors upon a video stream passing from a camera worn on the operative to the HMD after augmentation. A central wearable computer processes incoming encoded video from the camera unit, adding additional information from an internal source and/or transferred from an external wireless source. Hardware chroma-keying for this purpose allows video-rate display to be approached. The video is retransmitted in encoded form to the display device.
For a wearable AR system, energy consumption is important because batteries are carried upon the person and cannot easily be replaced in stressful scenarios. In [11], it was reported that there is approximately a linear relationship between bitrate and energy consumption and in [12] it was shown that transmission accounts for more than a third of the total energy consumption in communication on a mobile device. Therefore, reducing data transmission is an effective way to reduce energy usage. In packet-based rateless coding, each packet stream certainly contains packets, where is a small fractional overhead, typically amounting to two extra packets for values of k over 1000 [10], to ensure with high probability that all k information packets are decodable if received without error (rateless codes are constructed in probabilistic fashion). Raptor codes [13] have constant time encode and linear decode computational complexity, though additional precoding is performed prior to formation of the rateless code. Though it is outside the scope of the paper, use of rateless codes also will help reducing energy consumption at the transmitter, the transceiver at the AR processor and the receiver for the HMD, as their coding complexity is linear, compared to Reed-Solomon (RS) erasure codes for which coding complexity can be quadratic, , though decoding in the frequency domain reduces to .
A wide variety of biosensors have been proposed such as the emotion mouse [14], eyetrackers [3], oxygen saturation meters [15], and heart-rate monitor or electrocardiography (ECG) [16]. These measure cognitive load indirectly by the physical and emotional states of an operative. Direct information may come from monitoring the brain’s activity-electroencephalography (EEG) [17]. If a person is instrumented by sensors of these kinds, it is important [18] to reduce the wiring in order to improve the comfort of wearing what is in effect a wearable computer system, with display at the HMD, input from the various sensors and interpretation at a central processor unit. In the case of augmented group cognition then input/output will take place with nearby personnel. These considerations suggest a wireless sensor network [18] which is worn upon the body. In this paper, such a network is modeled for the case of a fireworker with an HMD for which the video input is moderated at the point of augmentation, according to input from ECG (physical input) and EEG (direct input).
1.3. Wireless Network Technology
In this system, both the encoded video stream from the camera and the EEG will potentially load the wireless sensor network, which is why in [18] a Bluetooth (IEEE 802.15.1) wireless network [19] was considered rather than the lower bandwidth Zigbee (IEEE 802.15.4) network [20]. Both are lower powered relative to WiFi (IEEE 802.11a,g,n) [21], with Zigbee having a lower duty cycle than Bluetooth. (Bluetooth has typical current of 0.3–350 mA, compared to IEEE 802.11’s 480–700 mA.) Bluetooth and Zigbee support a star topology network with a central privileged node. However, in a Bluetooth network or piconet with up to eight active nodes (an extension mechanism allows the construction of scatternets, with more nodes when necessary), access to the wireless channel is by time division multiple access (TDMA), whereas Zigbee access is distributed, which can lead to unpredictable delays, which will have an adverse effect on delay-intolerant video. The small user payload of only 101 bytes could also create problems if error resilence methods were applied to video data. A Zigbee network’s maximum (shared) bitrate is 250 kbps, whereas for Bluetooth v. 2.1 with enhanced data rate (EDR) [22], according to modulation type and channel conditions, the gross (shared) air rate is 3.0 Mbps which equates to 2.2 Mbps mean gross user payload. In [18], the bandwidth of a 128-channel EEG is assessed at 2 Mbps (1 kHz at 16-bit sampling [23]) and an ECG at 6 kbps (512 Hz at 12-bit sampling [24]). We suppose in this paper that either compression of the EEG signal has occurred or a reduced number of sensor patches are applied. In [25], compression was already applied to ECG signals, and we assume the bitrate of this source also is reduced as a result compared to the figure quoted in [18]. Notice that wireless EEG already exists in [26] with a 32-channel system for epilepsy monitoring rather than a wearable system.
Wireless networks are now preferred rather than the pioneering but cumbersome wired networks on older wearable computers [27]. Though a near-field intrabody network [28] may be ideal for low-power sensor/actuator devices, they do not have sufficient bandwidth for augmented cognition using EEG and indeed for video transmission. Bluetooth is similar in topology, frequency-hopping spread spectrum (FHSS), and modified TDMA to BBN’s BodyLAN [29], which was originally designed as a body network. Fabric area networks (FANs) [30], through near-field communication at connection points between wired items of clothing, reduce the problem of interference, a detraction of some systems. However, commercial difficulties have impeded the implementation of FANs. Therefore, Bluetooth is the most feasible wireless system currently available for this application, and in this paper we model the AR encoded video stream as a Bluetooth piconet. Bluetooth is very suitable for application of rateless coding as there is automatic feedback to the sender and because of its short range (typically for class 2 devices less than 10 m) that feedback is of low latency. In Bluetooth, fast ARQ is available by virtue of TDD polling, which is necessary for transmit/receive recovery, allowing a single-chip implementation. Bluetooth packets also automatically contain a cyclic redundancy check (CRC) for the user payload, allowing detection of a failed decode. It is assumed in this paper that the CRC is applied to the payload after rateless decoding.
The remainder of this paper is organized as follows. Section 2 presents essential background to the understanding of the simulated results. Section 3 introduces the proposed block-coding version of rateless channel coding, detailing the algorithm employed. It also sets out the scenario modelled taking into account the video stream and other traffic from biosensors and an external feed with additional augmented information and/or biosensor information from other operatives in the event of augmented group cognition. Section 4 presents our results in terms of packet loss due to buffer overflow or inability to repair packets in error before their display deadlines expire. Relative energy consumption and delivered video quality are presented. The section also considers packet arrival latency. Finally, Section 5 draws some conclusions.
2. Background
In this paper, rateless codes are employed to protect the AR video stream with minimal overhead. This section outlines the essential features of rateless channel codes, previous work in this field, and then goes on to describe the Bluetooth wireless network.
2.1. Rateless Codes
Rateless coding is ideally suited [7] to a binary erasure channel in which either the error-correcting code works or the decoder fails and reports that it has failed. In erasure coding, all is not lost as flawed packets may be reconstructed from a set of successfully received packets (if sufficient of these packets are received). Unlike fixed-rate erasure coding, rateless coding relies on feedback. An RS erasure code over an alphabet (where L is the number of bits in a packet) has the property that if any k out of the n packets transmitted are received successfully then the original k packets can be decoded. However, in practice not only must n, k, and q be small but also the computational complexity of the decoder is of order . Erasure coding in video communication has normally been applied to packets (refer to Section 2.2), but can be applied to any symbols, such as blocks within a packet. In the literature, the term block is sometimes used synonymously for packet but in this paper we reserve the term to blocks within a packet.
The class of Fountain codes [7] allows a continual stream of additional packets to be generated in the event that the original packets could not be decoded. It is the ability to easily generate new packets that makes Fountain codes rateless. Decoding will succeed with small probability of failure if any of packets are received. In its simplest form, the packets are combined in an exclusive OR (XOR) operation according to the order specified by a random low-density generator matrix and in this case, the probability of decoder failure is , which for large k approaches the Shannon limit. The random sequence must be known to the receiver but this is easily achieved through knowledge of the sequence seed. Luby transform (LT) codes [31] reduce the complexity of decoding a simple Fountain code (which is of order ) by means of an iterative decoding procedure, provided that the column entries of the generator matrix are selected from a robust Soliton distribution. In the LT generator matrix case, the expected number of degree one combinations (no XORing of packets) is , for small constant c. Setting ensures that by sending packets these are decoded with probability and decoding complexity of order .
Furthermore, if the packets are pre-encoded with an erasure code a weakened LT transform can be applied to the packets and their parity packets. The advantage of this Raptor code [13] is a decoding complexity that is linear in k. Notice that an essential difference between Fountain erasure codes and RS erasure codes is that Fountain codes in general are not systematic and that even if there were no channel errors there is a very small probability, assuming correct design, that the decoding will fail. In compensation, they are completely flexible, have linear decode computational complexity, and generally their overhead is considerably reduced compared to fixed erasure codes. In the 3GPP standard, a systematic Raptor code is arrived at [13] by first applying the inverse of the inner LT to the first k symbols before the outer precoding step.
2.2. Related Work
Rateless codes are now attracting applications in video streaming applications. In video streaming for cognitive radio [32], rateless error coding compensates an opportunistic secondary video packet source from interference by the primary occupant of the wireless channel. In essence, this is the same network coding technique as applied in [33], because it allows a set of subchannels distributed across the available wireless spectrum to stream scalable video without coordination between the sources. In [32, 33], the rateless code was applied to packets and not blocks within the packets, as also was the application to unicast video streaming over the Internet in [10]. Because the symbols of a Fountain code are generated from a sparse distribution, any uncoordinated sources are unlikely to construct the same two symbols. However, this is not how the present paper proposes to employ rateless coding, as in [32, 33] there are multiple uncoordinated channels, whereas herein there is a single channel that is coordinated with the receiver. For the same reason, BlueTorrent, concerning which [34] mentions in passing network coding for Bluetooth, is not related to the current paper.
In [35], it was observed that classic error control methods work poorly in terms of energy conservation, in line with similar comments in Section 1. It was proposed in [35] that the channel should be probed to find the error conditions, whereupon the level of ARQ retransmissions is adjusted. However, the volatility of the wireless channel may make measurements unreliable. The work in [36] proposed a scheme of error control which varied according to the channel conditions and to the relative energy budget for RS coding and selective repeat ARQ. As in our paper, a two-state type model allowed (Rayleigh) fading conditions to be modeled, in way that is independent of packet size. It was found that there was a threshold, beyond which FEC was necessary, despite the increase in energy budget. In [37], packet-level FEC (not block-level as in our paper) and power allocation are jointly optimized across cellular radio. The work combines layered video coding with FEC, with the degree of protection varying according the priority of the layer. The layers actually transmitted depend on the power resources of the sender.
In [38], rateless coding is selected for reasons of reduced decode computational complexity in an energy reduction scheme for wireless mesh networks. This scheme is compared to network coding and similar schemes for data broadcast. Others have noticed the advantage of rateless coding for energy conservation, for example in [39], rateless coding is applied in a sensor network context but for data not video and from the reduced decode complexity point-of-view and not necessarily because of reduced transmission overhead.
2.3. Bluetooth
Bluetooth’s short range, FHSS transmission, centralized multiple access control through TDMA and master-driven TDD system means it is less prone to interference from other Bluetooth networks. Bluetooth employs variable-sized packets up to a maximum of five frequency-hopping time-slots of 625 microseconds in duration. Every Bluetooth frame consists of a packet transmitted from a sender node over 1, 3, or 5 timeslots, while a receiver replies with a packet occupying at least one slot, with the result that each frame has an even number of slots. Bluetooth v. 2.1’s EDR supports gross air rates of 3.0 Mb/s and 2.0 Mb/s, in addition to the original 1.0 Mb/s basic rate. EDR allows higher quality video streaming, while a choice of channel rates allows the lower rate to be employed in adverse channel conditions, retaining the basic channel rate for packet headers and very poor (<10 dB signal-to-noise ratio (SNR)) channels.
Because of packet quantization effects, the Bluetooth asynchronous connection-less (ACL) packet sizes become significant. From Figure 1, it is apparent that selection of a small packet size results in relatively low throughput, and fully-filled packets are favored over partially-filled packets. Therefore, we assume that a Bluetooth controller’s behavior is that, given a maximal Bluetooth packetisation scheme, packets up to the maximum user payload will be formed. However, if the arriving packets do not justify the preset maximal scheme a reduced scheme is applied. The packet structure of Bluetooth is further described in the course of the following section.
3. Methodology
This section now describes our block-based rateless coding system and then goes on to describe the scenario that is simulated.
3.1. Redundant Block Transmission Algorithm
Figure 2 shows the partition of a video-bearing Bluetooth packet payload into three parts: (1) a variable-sized redundant block portion, with the blocks within this portion generated by the rateless algorithm from prior packets; (2) the data of the next packet divided into blocks with an additional blocks generated by the rateless algorithm, as blocks are required for reconstruction of the original k blocks with high probability; (3) a CRC which is a default part of a Bluetooth packet but which we assume is applied to the decoded k blocks of the current packet. Upon failure of the CRC, additional blocks are requested from the sender and these are sent in the first part of the next packet together with any other blocks from yet to be reconstructed packets.
The complete block-based rateless coding algorithm is described in detail in Figure 3. Assume initially that just the one prior packet has failed then redundant blocks are now piggybacked upon the current packet to add to the original blocks already transmitted to increase the probability of a successful decode. After an attempted decode, the CRC of that prior packet is applied to establish whether there has been an erasure. If there is an erasure additional blocks are requested through Bluetooth’s fast ARQ mechanism, unless the duration of block retransmissions already exceeds the display deadline of the video frame of which that packet’s data forms a part. The display deadline in the simulations was set to a constant d number of retries.
Critical to the operation of rateless error correction is the number of blocks contained in part 1 of a Bluetooth packet payload. Bluetooth’s native block-based FEC scheme employs 15-bit blocks as a consequence of its error coding scheme (refer forward to Section 3.2) and, for comparison and ease of implementation, 15-bit blocks are also employed in the block-based rateless coding scheme. If redundant blocks are to be sent then a minimum and a maximum number of 15-bit blocks is defined, being 5 and 50, respectively, in the simulations of Section 4. Leaving aside initialisation packets, the starting number of redundant blocks was the minimum number (five blocks) in our simulations. Upon receipt of a consecutive sequence of n successfully-transmitted packets, 100 in the simulations, then the limit is reduced by one. Upon a failure to reconstruct any packet after the dth transmission of its blocks the number of redundant blocks included in a packet is increased in the future by a factor α, set to 1.5 in the simulations. This conservative policy for a volatile channel results in a rapid increase in redundancy when uncorrectable errors first occur.
If more than one prior packet of the same frame type has errors then the redundant block allowance is split according to the proportion of retransmissions remaining for each packet, allowing for some irregularity due to the need to apportion an integer number of blocks. The ratio is calculated as a proportion of factor d. A simple acknowledgment of the differing importance of video frame types was made by altering the allocation in the ratio 3:2:1 for I-, P-, and B-frame packets, respectively. (I- and predictively code P-frames are reference frames (refer to Section 1) while bi-predicted B-frames themselves have no predictive role.) Other priority-based schemes are possible.
3.2. Other FEC Schemes
Bluetooth already has FEC-bearing data medium (DM) packets, available at the basic rate of version 1 in the event of poor SNR. The Bluetooth system of FEC and error control is already extensively described in [22] and summarized in [19]. To avoid extending the length of this paper, the reader is referred to those documents, while this section describes variants of the Bluetooth FEC system. An expurgated (15, 10) Hamming code is applied to 15-bit blocks and can cope with burst sizes of two, depending on decoder [40]. As a point of comparison with rateless codes, it is supposed that the DM packet scheme is extended to the EDR transmission modes. This extension scheme has already been proposed in [41] and is entirely feasible. Table 1 summarizes the additional EDR asynchronous connection-less (ACL) mode packet types currently available (according to the specification), as well as EDR DM-type packets in the event that symbol-level FEC were to be added to EDR.
In the simulations of Section 5, the FEC-bearing packets are the 3DM packets from Table 1. Additional comparison is made with an adaptive FEC-bearing scheme that assumes perfect channel knowledge. FEC-bearing packets are only selected when the channel enters a bad state. The adaptive scheme is introduced as it has the ability to save energy by reducing the overhead when channel conditions ease. Because the native FEC-bearing Bluetooth scheme already has a rate of 1/3, that is, considerable overhead, automatic ARQ is normally turned off to avoid increasing the FEC overhead. (ARQ is effectively turned off [22] by setting the Bluetooth flush timeout to a minimal value.)
In addition, a comparison is made with a simple packet-based rateless coding scheme in which 10% redundancy is added (i.e., one in ten packets is redundant), compared to 33% redundancy for fixed FEC. The block-based rateless and the adaptive FEC schemes do, of course, have a varying percentage of redundancy.
3.3. Channel Model
A Gilbert-Elliott two-state discrete-time ergodic Markov chain models the wireless channel error characteristics between a Bluetooth master and slave node. The mean duration of a good state, , was set at 2 seconds and in a bad state, was set to , where a is a parameter which is varied to alter the duration of bad states. In units of 625 microseconds (the Bluetooth time slot duration), which implies from that, given the current state is good (g), Pgg, the probability that the next state is also g, is 0.9996875. Both good and bad states are modeled by a Rayleigh channel with the mean SNR being dB and dB in the g and b states, respectively.
3.4. Wearable Network
The Bluetooth network in Figure 4 contains the two biosensors, the video camera source and the HMD, along with an external source which may act as a means of exchanging biosensor data with other operatives and as a source of external sensor data. The video source is assumed to be of variable bit-rate (VBR) to ensure higher quality within the restrictions of the available bandwidth. The encoded video is transferred to the central node where, after decoding, augmentation of the display takes place along with moderation of that display in line with interpretation of biosensor data. Notice that if the display contains text then good resolution is needed. The EEG and ECG biosensors are assumed to output at a constant bit rate (CBR). The external source was modeled as an on-off source in the ratio 1 second on to 2 seconds off with its bitrate divided equally in the two directions. However, polling packets from the master node and null return packets to the external source will occupy a significant portion of the available bandwidth when the external source is off. The assumed data rates of the sources are detailed in Table 2 along with packet sizes. The rate for the ECG source is taken from [42]. ECG, EEG, and the external source all used 3DH5 packets (refer to Table 1). For video, the payload size is determined by the amount of redundant blocks required for correction of the prior failed packets. Thereafter, 3DH5 packets are selected.
3.5. Simulation Setup
This research employed the University of Cincinatti Bluetooth (UCBT) extension to the well-known ns-2 network simulator (v. 2.28 used). The UCBT extension supports Bluetooth EDR but is also built on the air models of previous Bluetooth extensions such as BlueHoc from IBM and Blueware. The Gilbert-Elliott channel model was coded in C++ to be called by an ns-2 otcl script. All links were set at the maximum EDR 3.0 Mbps gross air rate. The UCBT simulation parameter for antenna type was set to omnidirectional, and for the distance between devices was set to 1 m, as is appropriate for a wearable network. (Experiments were also conducted with the distance set 2 m with no noticeable difference in the results.) All other settings were the default ones. Simulation runs were each repeated 100 times, and the results averaged to produce summary statistics.
The simulations were principally carried out with input from an MPEG-2 encoded bitstream at a mean rate of 256 kbps for a 30-second video clip with moderate motion. PSNR was found by reconstructing with a reference MPEG-2 decoder. The display rate was 25 frame/s, resulting in 750 frames in each run. The source video was common intermediate format (CIF)-sized ( pixels) with a GOP structure of , and (when in standard codecs designates the GOP length and is the number of pictures between anchor pictures). In [43], it was demonstrated that forming fully-filled Bluetooth packets outweighed the need to preserve MPEG-2 slice boundaries, which over the fixed Internet are preserved for error-resilience purposes. Therefore, as mentioned previously, 3DH5 packets are selected by default and, thereafter, fully-filled packets of the 3DH type are formed from the arriving encoded video stream. The results of simulating the video stream under these circumstances are now described.
4. Simulation Results
Experiments were conducted streaming the video of Section 3.5. Metrics were recorded across both hops in Figure 4 over which video was streamed. For example, packet loss is recorded as a total across both hops. A varying number of redundant blocks were included in the packet payload if one or more prior packets were found to have failed. For any one packet in error, retransmissions continued until the number of retransmissions, d, exceeded ten, assuming that after ten attempts at reconstructing the packet the display deadline would be exceeded. At a frame rate of 25 frame/s, a frame is displayed every 0.040 seconds, while ten retransmissions take 0.375 seconds. Assuming a worse case of each of 18 slices in an MPEG2 CIF-sized frame then a small playout buffer of about 20 frames is adequate even if all 18 were in error. A large buffer would result in passive energy drain, reducing the lifetime of the system before recharging is necessary. After d is exceeded then the packet is declared as lost. A send buffer size of fifty packets was sufficient to avoid packet loss by buffer overflow, though increasing the video arrival rate could change that.
Figure 5 shows how there is a sharp reduction in the packet loss ratio (the number of lost packets to the number of packets transmitted) at a given average SNR for a relatively small investment in redundant blocks. All loss occurred through exceeding the retransmit limit. Notice also that in Figure 5 that there is a single-state Rayleigh channel, whereas later tests use the two-state channel model of Section 3.3. Figure 6 shows that as the retransmission depth, d, increases then there is a higher chance of recovering a previously failed packet, as packet losses decline with d. However, cannot be chosen arbitrarily as it can lead to missed display deadlines. Ideally, d should match the playout buffer size, as an arbitrary choice can lead to missed display deadlines.
In Figure 7, the augmented cognition traffic and the other traffic sources from Table 2 are turned on, while the packet loss ratio for each of the other FEC-bearing schemes of Section 3.2 is compared with the proposed rateless coding scheme. The packet loss ratio is the ratio of packets lost against total packets transmitted in the video streams. The loss ratio is adjudged against worsening channel conditions as regulated by parameter a from Section 3.3. From Figure 7, it is apparent that the proposed rateless scheme outperforms the native schemes and increasingly so as the bad state durations increase. In comparison to a simple packet-based scheme, the block-based rateless scheme reduces packet loss because it is able to adapt to increasing burst lengths. The packet-based rateless coding scheme has a reduced level of redundancy compared to the fixed FEC scheme, resulting in more packet loss. The adaptive FEC scheme is less efficient at preventing errors compared to the packet-based rateless scheme but is better able to compensate when the channel parameter increases.
The various schemes were also compared, see Figure 8, in terms of energy efficiency. The energy efficiency is the delivery ratio adjusted according to the transmission power. However, assuming the transmission power is normalized to one, the energy efficiency is the same as the delivery ratio, that is, the ratio of the data successfully transmitted to the total data transmitted. Though the “no FEC” plot involves no overhead from FEC, it still has a poor energy saving efficiency compared to the proposed scheme because of the fewer bits transmitted successfully. Adaptive FEC is relatively better at energy reduction than fixed FEC but, of course from Figure 7, the number of unrecoverable packets is greater. In better channel conditions, block-based and packet-based rateless coding have similar energy efficiencies. However, when the channel conditions worsen, fewer packets are successfully received through the packet-based variant, causing the energy efficiency to deteriorate sharply.
According to the AR scenario, packets make two hops across the Bluetooth network before display on the HMD. Processing delay at the master node is neglected. The data rates are the same as those in Table 2. A playout buffer of size fifty packets is assumed, which is the same size as used in [44] for a mobile application. Larger playout buffers lead to memory drain through passive energy consumption. In these tests, ARQ is enabled but set to a maximum number of four retransmission requests, while the blocks in the rateless scheme can be resent up to ten times. From the figure, fixed FEC packets suffer from considerable delay when the channel conditions worsen, as they must be retransmitted when error correction is insufficient. Adaptive FEC fares better as the level of FEC can be adjusted to cater for the longer error bursts. However, though in the proposed scheme blocks may be repeatedly piggybacked upon outgoing packets, they miss fewer display frame display deadlines, which is the relevant criterion for video transmission. The impact on packet arrivals is examined in Figure 9. The figure records the percentage of packets that miss the display deadline for the block-based rateless and FEC schemes when the channel bad state duration increases in duration.
In Figure 10, a comparison is made between the delivered video qualities for selected bad state durations for which PSNR is of a reasonable level. The figure shows that in terms of delivered video quality the rateless scheme also outperforms the Bluetooth FEC scheme when applied to EDR packets. The relative improvement increases with lengthening bad state duration.
Figure 11 is a comparison between the overhead incurred by the proposed block-based rateless scheme (with 12 redundant blocks) and a simple packet-based rateless coding scheme. A single-state Rayleigh channel is simulated as also occurs in Figure 5. Both schemes must sustain the small percentage (10%) overhead from a finite length rateless coding scheme mentioned in Section 2.1. Figure 11 shows that the overhead of the proposed scheme remains reduced at much lower SNRs compared to the simple packet-based system. The channel conditions are responsible for the sudden increases in overhead (just as in Figure 5 in which there are rapid increases in the packet loss ratio). Though the effect is exaggerated in these conditions, the gain in reduced overhead is apparent.
5. Conclusions
Augmented reality is normally associated with wearable computers. It is now feasible to provide wearable wireless networks with sufficient bandwidth capacity to allow a video stream from camera to head mounted display. An important class of wearable computer users is those that can be broadly termed emergency workers. As these operatives often work under stressful conditions, it becomes necessary to monitor the worker with biosensors. Some of these sensors such as EEG have high bitrates leading to delay for other traffic and reduced video data rates. Therefore, it is important to model the wearable network, which we have done by considering the video stream from camera via the CPU, where augmentation takes place, to the display. Some form of forward error control is required, because a wireless channel is also prone to interference and noise and because lost packets have an enduring effect upon the compressed video stream (until the next intrapicture refresh point). We have proposed block-based rateless coding, which, from the paper’s results, compared to block-based forward error correction variants can jointly improve energy consumption, packet arrival latency, and delivered video quality. Block-based rateless channel coding is well suited to a Bluetooth wireless network, because feedback to the centralized packet scheduler occurs immediately and with little propagation delay. Simple packet-based rateless schemes are less suited, as without feedback they are unable to adjust to worsening channel conditions, despite advantages in coding efficiency. However, to construct a scheme for block-based rateless coding requires recursive piggy-backing of redundant blocks onto outgoing packets. This is a cross-layer approach, whereas simple packet-based rateless coding is applied at the application layer. Nevertheless, this paper reports an implemented scheme that results in around 5 dB gain in video quality in poor channel conditions.
Acknowledgment
This work was supported by the EPSRC, UK, under Grant no. EP/C538692/1.