Abstract
Packet-level Forward Error Control (FEC) for video streaming over a wireless network has received comparatively limited investigation, because of the delay introduced by the need to assemble a group of packets. However, packet-level interleaving when combined with FEC presents a remedy to time-correlated error bursts, though it can further increase delay if this issue is not addressed. This paper proposes adapting the overall degree of interleaved packet-level FEC according to the display deadlines of packets, transmit buffer occupation, and estimated video input to the wireless channel, all of which address the issue of delay. To guard against estimation error, the scheme applies a conservative adaptation policy, which accounts for picture type importance to ensure that display deadlines are met, thus avoiding this defect of interleaving. The paper additionally introduces a greedy algorithm that effectively groups packet-level FEC protection according to packet priority. Priority encoding adds extra protection during deep fades. As feedback is not required, the interleaving scheme is suitable for all forms of video broadcast. A Bluetooth piconet demonstrates the packet-level FEC interleaving scheme, which provides higher quality delivered video compared to the industry-standard Pro-MPEG Cop#3r2 interleaving scheme.
1. Introduction
In a wireless network, the channel is prone to radio frequency noise together with various forms of interference including multipath-induced fast and slow fading. As a result, wireless networks suffer both from high bit error rates and high packet error rates, depending on the quality of the wireless link. Additionally, packet loss from buffer overflow as a result of congestion still occurs on a wireless network as it does on a wired network. Because video streaming is a real-time service, packet drops can also occur through too late arrival of packets at the decoder. Application-layer Forward Error Correction (FEC) can be applied to symbols within a packet (blocks, individual bytes or bits). We refer to this form of application-layer FEC as subpacket-level FEC. It is also possible to apply FEC to complete packets, treating each packet as a symbol, which in this paper we refer to as packet-level FEC, as is also common practice [1] in the literature. Packet-level FEC at the application has the ability to correct packet loss from channel error, congestion, and packet delay, though this paper concentrates on packet loss arising from channel error.
Wireless interference often manifests itself as error bursts correlated in time. Unfortunately, lengthy bursts of errors, possibly extending over several packets are now accepted [2–4] as being more harmful to received video quality than randomly occurring errors. FEC applied at the subpacket-level is suitable for correcting short-term fading and additive white Gaussian noise but is less suited to combat long-term fading [1]. The same observation applies to link layer retransmissions: they can cope with fast fading but are insufficient in combating slow fading. To further combat noise and interference, subpacket-level FEC can be built in at the data link layer and be combined with packet-level FEC at the application layer. In this arrangement, the application-layer code acts as an outer code that is concatenated with the FEC in the lower layer. In general, employing packet-level FEC as a complement to sub-packet-level FEC has the advantage that broadcast as well as unicast video streaming can be supported without changing the form of error control. Because of anticipated high packet error rates, in the 3GPP’s Multimedia Broadcast/Multicast Streaming service (MBMS) [5] packet-level FEC was introduced. Nonetheless, MBMS employs built-in, data-link layer, packet-level FEC rather than application-layer FEC.
However, the main thrust of our paper is to strengthen application-layer packet-level FEC by combining it with packet interleaving. Application-layer FEC can take advantage of the known structural characteristics of a video stream. Because the delay characteristics of a video stream can be known in advance, it is additionally possible to reduce the impact of packet interleaving upon delay.
An important contribution of this paper is that, knowing packet display deadlines and an estimate of delay arising from already buffered (and interleaved) packets, it is possible to adapt the extent of interleaving to avoid missed deadlines. Though a number of packet-level FEC schemes exist [6–8], the delay introduced by additional interleaving runs the risk of missed frame (picture) display deadlines at the decoder. It is true that whether the packet transmission order is altered or not, FEC cannot be applied until a packet interleaving matrix is assembled. However, excessive delays may be avoided. For video, the display deadlines of video packets are knowable from packet headers and may be provided to the data-link layer through cross-layer interaction. Therefore, we introduce an adaptive interleaving scheme for video streaming.
It also seemed natural to us to extend our delay-aware FEC interleaving scheme to unequal FEC protection according to picture type. Priority-Encoded Transmission (PET) [9] is a scheme in which packets receive an increased number of FEC packets according to their importance. (The PET scheme in [9] is general but does include a priority option based on video picture type.) However, PET does not take delay into account, whereas putting multiple FEC packets within an FEC subgroup into an interleaving matrix may add to that delay.
In the extended PET scheme contributed by this paper, a constant-sized FEC allowance per FEC subgroup is applied, while the number of information packets per FEC subgroup is varied, which is the reverse of the way PET has previously been applied [9]. Our paper presents a detailed algorithm for matching up packets to appropriate FEC subgroups. As the number of ways of matching packets to an FEC subgroup may be NP-hard in complexity, we have devised a greedy algorithm of lower complexity which still results in a worthwhile improvement in video quality (luminance PSNR), depending on error burst length, over a nonpriority-based, packet-level, interleaved FEC scheme. We report similar gains over the industry standard Pro-MPEG Cop#3r2 interleaving scheme [10].
A nonbinary code such as Reed-Solomon (RS) can repair packets affected by long error bursts. FEC applied at the packet-level may also repair individual symbols within each packet, irrespective of the number of single bit errors within that symbol. The main detraction of RS codes is decoder complexity which for syndrome-decoding is and for frequency domain decoding is , with trade-offs depending on block size [11]. In fact, Raptor rateless codes [12] as used in MBMS [5] have complexity for encoder and decoder alike and could be substituted for RS codes. Moreover, if packet-level FEC is applied at the same coding rate as was formerly applied through sub-packet-level FEC, then the data rate remains the same. However, the proposed extended PET scheme undoubtedly does have implications in terms of computational complexity, which the developer should consider. In the algorithm of Section 2.5, most operations are linear in time, , except for sorting the packets which has a complexity of , for a heap, sort, or quick sort; but this type of analysis underestimates the practical difficulties of implementing a scheme and the delay introduced while the calculations are made. There is also a bandwidth implication arising from additional FEC packets, though this overhead has been accepted for the gains that result in 3GPP’s MBMS scheme [5].
To increase the effectiveness of packet-level FEC, it is sensible to apply it across an interleaved group of packets, because in that way consecutive packets within the error burst are separated out. Packet-level interleaving is likely to be helpful during a deep fade when the Carrier-to-Noise Ratio (CNR) is low (10?dB). It may also be concatenated with block-error coding within each packet, that is, combined with sub-packet-level coding. There is another point to notice. It is unnecessary to alter the packet transmission order from that of the data arrival order when applying FEC to interleaved groups of packets. Instead, the temporal extent of separation between packets within the same FEC group determines the protection afforded by interleaving.
The research in [3] reports an influential model for the differential impact on video quality of packet losses, depending on packet burst length. Consequently in [13], simple packet interleaving (without FEC) was applied. In all cases, physically interleaving the packets resulted in an improvement in quality, though it did not follow that continuing to increase the amount of interleaving resulted in continued improvement of the received video quality. In [13], the level of interleaving is constant and does not dynamically adapt to packet latency. In [14], physical packet-level interleaving was taken further by equably distributing the video packets according to their content importance, specifically the impact of their loss on error propagation.
In [15], FEC is applied to packet-level interleaving with the degree of interleaving dependent on the size of an MPEG-4 Video Object Plane (VOP). In [15], the packet arrival order is replicated in the transmission order, but FEC packets are formed by a virtual interleaving of the packet stream. Though a packet copy buffer appears to have been used to allow continual streaming, this will still cause delay at the decoder in the event of a packet loss, whereas delay factors do not appear to be taken into account in [15]. Therefore, a weakness of [15] appears to be that it does not take display deadlines into account.
The amount of FEC protection can be varied depending on picture importance. Priority Encoding Transmission (PET) for multicast video was introduced in [9]. In [9], probabilistic erasure-resilient codes are employed to reduce decoder complexity. In [16], PET is applied at the byte level for transmission of progressively encoded still images. This is a form of sub-packet-level rather than packet level interleaving. The same remark applies to earlier applications of PET to video such as in [17]. Moreover, these PET schemes vary the FEC redundancy, rather than keeping the level of FEC constant, and vary the number of packets protected.
Mention should be made of the important practical packet interleaving scheme Pro-MPEG Cop#3r2 [10]. This scheme is intended for MPEG-2 Transport Stream (TS) payload packets, which are of constant 188 byte length, resulting in 7 packets per RTP/IP packet. A 2-dimensional matrix of MPEG-TS payload packets is protected by an additional FEC packet at the end of each row of payload packets and likewise an additional FEC packet at the end of each column. If packets are encapsulated in the MPEG-TS, the additional row of FEC packets guards against the possibility of the loss of more than one packet in a column, as there is an additional means of reconstruction. The scheme scales according to the matrix size but does not appear to be payload priority aware. In [10], it is tested in a fixed Internet scenario, with a uniform distribution of packet loss errors, rather than for wireless conditions.
To test the validity of our ideas for robust video streaming a Bluetooth case study is presented in Section 3. In an IEEE 802.15.1 (Bluetooth) [18] piconet, a broadcasting master node does not receive acknowledgments. By default it simply repeats the broadcast a preset number of times. In this network, if packet-level FEC were to be implemented, then it would no longer be necessary to rebroadcast delay-sensitive video data. “Push” video broadcasting, as well as unicast IPTV, represents an opportunity for deployment of a Bluetooth service in coaches, trains, cars, and other hotspots. As an example of an existing in-vehicle system, TETRA is the leading wireless cellular system for the emergency services and in combination with TETRA Bluetooth video transmission allows the stream to be transferred within an emergency vehicle (fire-engine, police car, ambulance, etc.). In [19], application-layer FEC is applied at the sub-packet-level by replacing padding bits with FEC bits. In a Bluetooth v.1 case study, embedded FEC (using RS coding) is concatenated with built-in data-link layer FEC. Though this form of FEC could be extended to packet-level FEC, the authors of [19] preferred to reduce latency. Therefore, our packet content-dependent scheme could act as an extension to the work in [19].
The remainder of this paper is organized as follows. Section 2 details the priority-based FEC and interleaving scheme introduced by this paper. Section 3 prepares the way for the results reported in Section 4 by means of a Bluetooth case study. Finally, Section 5 draws some conclusions.
2. Interleaved FEC Scheme
2.1. Packet-Level Protection
We suppose that if a packet has bits, then a codeword is taken from the extension field GF according to RS erasure coding, assuming that is greater than the number of arriving packets, . If GF is the th packet data of length , then FEC packets are formed from polynomials of the form using powers of , a primitive root of the Galois field GF, as the variable in (1). Then, by sending packets, any?? correctly received packets serve to reconstruct the original packets. In fact, the current preference is not to form the FEC packets by direct application of (1) but by means of a generator function. It is known that no other code can recover packets from fewer received packets; that is, RS is a maximum distance separable code.
In packet interleaving, each column of the interleaving matrix is formed by packets, while the packets are read out across the rows of the matrix. Figure 1 illustrates the scheme, showing FEC applied along the columns of the matrix. RS coding takes place for each packet data or symbol horizontally along the packets. For each set of symbols, if the syndrome vector evaluates to zero, then it is known that the packet error(s) did not occur in that column of symbols. When a nonzero syndrome vector (of length ) is found, then error correction takes place using the RS code with the aid of an error location polynomial. is the total number of data packets allowable by the delay-aware scheme. The numbering of packets gives the transmission order, which is the same as the data arrival order. Because FEC is applied across nonconsecutively transmitted packets, the FEC protection is better able to protect against a burst that affects more than one packet of the packet stream.
In general, with a larger decoding buffer, a greater number of packets can form an FEC group. Consider the maximum number of packets, , that can participate in formation of an interleaved FEC group of packets with a coding scheme satisfying a deadline constraint The overall packet delay consists of three components: the time that it takes for the input video packets arriving to form the FEC group, the time needed for transmission of packets that are already buffered in the transmitter buffer; and the transmission time needed for sending FEC grouped packets.
There might be an overlap between the time needed for arrival of all packets of the FEC group and the time required to clear the buffer of other packets. In order to satisfy a deadline constraint, the worst-case scenario should be taken into account when calculating the maximum packet delay. In other words, upon the arrival of the first packet of an FEC group, if the buffer fullness is relatively low, that packet will be more likely to find the buffer emptied before all packets of the group arrive. In which case, the overall packet interarrival times will be a dominant cause of delay. However, if the buffer fullness is already high, even after the arrival of all FEC group members, the packets will have to wait further until previously buffered packets are all transmitted.
In Figure 1, FEC is calculated over the packets of each column. In case of a data packet failure, the packet may not be recovered until the last packet of the same column is received. It is clear that, in case of low buffer fullness, the first packet of the FEC group (packet #1 in Figure 1) will experience the maximum delay, as it has to wait for all other data packets to arrive and all data and FEC packets until the packet at end of the first column (packet # ) is transmitted. Therefore, if the average packet interarrival time is represented by and the average packet transmission time (service time) is denoted by , with a packet-level FEC scheme ( original and redundant packets) the maximum FEC group size, , should be chosen to satisfy the delay constraint , That is,
The first term on the right-hand side of (2) refers to the time the first packet has to wait until all FEC group packets arrive, while the second term represents the delay due to transmission of all packets until the FEC packet at the end of the first column is transmitted. Simplifying (2) yields where floor() reduces its argument to the nearest lower integer lower. If the ratio of the average packet interarrival time, , to the output service rate, , is represented by (i.e., ) and the ratio of the maximum delay, , to the average service time is denoted by , , then (3) can be written as
Notice that stable operation of the system requires , which implies that . Figure 2 shows the 3D graph of versus and when a (2,3) FEC coding scheme is employed. As expected, decreases when the value of increases. This is because a higher value of implies a longer waiting time until all packets of the group arrive. In contrast, a larger deadline margin (higher value) allows a greater number of packets to form an FEC group.
Notice that the above analysis is true if the assumption of relatively low buffer fullness is valid. More specifically, the case is valid if the buffer is completely emptied before the arrival of all packets of the group: where represents the number of already buffered packets when the first packet of the FEC group arrives. Replacing the value of from (3) in (5) and ignoring the floor() operation yields, The above statementcan be rewritten in terms of and parameters as However, if the buffer is relatively full upon the arrival of the first packet of an FEC group, the time needed for clearing the buffer from packets that are already buffered is longer than the delay due to arrival of the FEC group packets, which in this case means Simplifying (8) yields which can be simplified to
Figure 3 shows the maximum FEC group size, , versus the number of buffered packets, , and . The figure illustrates the results for two different values of . In the situations shown, both scenarios have been considered: The area in red refers to the situation when delay due to formation of the FEC group (packet interarrival times) is longer than the time needed for the buffer to be emptied from other packets, in which case the value of is calculated from (4). Alternatively, the blue area represents the case when the packets have to wait further for the buffer to be emptied after the FEC group is formed. As expected, the group size is not affected by the number of buffered packets if the first scenario is the case (i.e., M???does not change with respect to B, for the red part of the graph). Also comparing Figures 3(a) and 3(b) shows that by increasing the value of , the second scenario will happen more frequently. This is because a higher value of implies that it takes longer time for the FEC group to be formed. It also confirms that the FEC group size can be enlarged when the average packets interarrival times are increased (and is decreased).
(a)
(b)
2.2. Delay Estimation
In order to estimate the extent of packet interleaving, an estimate of the video packet arrival rate needs to be made, allowing to be predicted. Adaptive linear prediction [20] is especially suited to real-time video [21], because it does not require prior knowledge of video statistics and does not require time stationarity.
The predicted rate is arrived at by a -order linear prediction filter (LPF). The -order LPF prediction filter is represented by where is a predicted value estimated from previous samples (packet arrival times), while the are the adaptive filter weights indexed by . The weights are estimated [20] through where is sample of the length column vector of weights and, is a length column vector of measurements over time: when represents the vector transpose. The variable is the error between the measured and the predicted value.
2.3. Conservative Estimation
As this process relies on an estimation of delay caused by cross traffic within a piconet, a conservative, robust algorithm [22] is necessary to avoid the consequences of an over- or underestimate. Given also that video picture types are not all of equal importance, the estimation precautions in our work are weighted according to whether the packets originate from I-, P-, or B-pictures. Within a Group of Picture (GOP) structure, Intracoded pictures influence the decoding of all other pictures in the GOP, typically 12 or 15 display frames, representing about 0.5 second to a GOP structure of 25 or 30?Hz (frame/s), respectively. Predictively-coded P-pictures contribute to the decoding of later pictures. B-pictures are bipredicted from adjacent reference pictures (I- and P-) and, hence, are of limited importance. More complex weighting schemes [23] are possible, but these may not be effective in real-time or may involve encoder intervention, which reduces their generality.
The weighting scheme was applied as follows. Two weighting coefficients are applied to the known display deadline for each packet. This display deadline value is assumed to be constant for all packets.
The first coefficient, , accounts for the position of a packet in an interleaved group of protected packets. Each row of packets within the group will experience different delays. For example, in Figure 1 the packet numbered 1 has to wait for other packets to be received if FEC functionality is to be applied. If there are different levels of coefficients: , each corresponding to a row of data packets, then if . As an example, we set for just two rows of packets.
The second weighting coefficient considers the picture type of the packet. The algorithm is more conservative when dealing with reference packets, that is, . In simulations, The settings for the are heuristic in nature but are similar to the relative weightings in [22], accounting for the consequences of an over- or underestimate.
Finally, a joint weighting of the common display deadline is formed by summing these weights across the group of interleaved packets to produce combined weighting : Notice that in (15) for packets that are in the same row of the interleaving matrix the values of will be the same, as previously stated.
A further important feature of the delay-aware scheme is “lazy” evaluation or last-minute determination of the number of interleaved packets in a group, . The decision is normally taken at the point in time when there are just enough packets in the transmit buffer to overlap with the time that it takes to construct the FEC. This procedure has an important advantage because it means that the various estimates required to form are given the maximum time to be right. Prematurely making an estimate of the arriving traffic rate might lead to an over-ambitious estimate of , leading to missed display deadlines.
2.4. Subpacket-Level Protection
We compare packet-level interleaving to bit-level FEC following the Bluetooth model. In the Bluetooth model, each packet of payload length, , is divided up into blocks, with each block constituting a systematic FEC codeword with ten data bits and five FEC-coded bits for a rate of . We assume that the header is immune to error since it is always transmitted at the reduced basic rate and is protected with a triple redundancy code. The generator for the Bluetooth (15, 10) binary Hamming code is that of an expurgated code, that is, and, hence, can only cope with burst sizes of two, depending on decoder [24].
As an additional point of comparison, we also consider physical interleaving [25] within a packet. To apply this form of interleaving, the bit sequence is read into the columns of a matrix with height 15, whereas bits are read out into a packet by rows of width . The result is that formerly adjacent bits are separated by distance , which distance is the burst-size immunity of this sub-packet-level scheme. For comparison purposes in the nonextended, packet-level FEC scheme, the same coding rate is retained as a point of comparison.
2.5. Priority Encoding Algorithm
Hitherto a constant level of FEC irrespective of content has been assumed. However, a compressed video stream consists of a hierarchy of information. In this section, a form of PET is applied to cater for that variety. Again for simplicity, the number of FEC packets, , is set to one, while the coding rate for each group of FEC protected packets is varied by changing . Calculation of FEC subgroups begins when packets have arrived; see Figure 4. To determine a global redundancy rate allowance, , at the time of the formation of the next global group of packets, set where is the estimated available bandwidth and is the estimated video arrival rate. In Figure 4, there is an adjustment to to form but for ease of description the origin of this adjustment is explained further on. This redundancy rate allowance, or , is now allocated between the picture types.
Set three constants for the number of I-, P-, and B-pictures in a GOP. For example, in a typical 12-picture GOP, these would be , and pictures. Now set variables, , and to the relative data size of each picture type, depending on a dynamically constructed histogram of those sizes recorded as pictures arrive prior to packetization. This allows size values to be formed as: Using these values, form a normalized size weighting for each picture type as: Estimate by some means picture type importance values. For example, values used by us in Section 4 were and
Now, solve for in Then, as Figure 4 now picks up upon, set the respective redundancy rates for each picture type as Estimates of the number of packets that might be associated with each picture-type packet in an FEC subgroup are now found as
Reassign the to a single variable , with . Sort the packets by their value of into a list of ascending order of value, so that I-picture packets are at the start of the list and any B-picture packets are at the end of the list. A greedy algorithm is applied in which the most important packet requirements are matched first by extending a packet matching window incrementally over adjacent packets. The starting size of the window is given for each of the packets by , which is the value of truncated or floored to the nearest integer. Within that window size as a starting point, the FEC requirement of the windowed FEC subgroup is matched as closely as possible to the value one, the preset value of , the amount of redundant packets, subject to a maximum subgroup size . The FEC requirements are given according to picture type by the and of (21). Thus, the FEC requirement is met already either by the aggregated requirements under the window or by extending the window a further one, two, or three packets, that is, for . Matching is by taking the absolute difference, in Figure 4, and the minimum difference is selected as the chosen match. The windowed packets form the FEC subgroup. Having established an FEC subgroup the algorithm repeats matching with the remaining packets in ascending order until all FEC subgroups have been formed. The final two subgroups are formed by comparing the FEC arrangements from various allocations of packets to these remaining packets.
Clearly, positive or negative surplus FEC value, the of Figure 4, will accumulate during matching. The summation of the surpluses is transferred to the next global group of packets to even out the overall rate over time. Some such transfer is inevitable in this scheme, as at any one time the packet types might be dominated (say) by -picture packets or (say) -picture packets. FEC subgroups form unevenly-sized columns within a transmit buffer. Thus, in (17) (see Figure 4) the global redundancy was actually adjusted to , with error being the negative or positive surplus FEC value accumulated in the previous round of packets.
Figure 5(a) is a pictorial example of the formation of the FEC subgroups. The arrived packets are sorted into ascending order by their associated packet number value, , before selection of windowed FEC subgroups is formed. In Figure 5(b), the FEC subgroups are arranged for the formation before transmission. The leading packets of each FEC subgroup are successively transmitted. Thus, physical packet interleaving takes place in the extended version of the scheme, and Section 4.2 reports the effectiveness of that interleaving.
(a)
(b)
3. Experimental Methodology
3.1. Bluetooth Packetization
Section 4 presents results from a Bluetooth case study. Bluetooth’s short range (less than 10?m for class 2 devices) and Frequency Hopping Spread Spectrum (FHSS) transmission means that it is less prone to interference from other Bluetooth networks. Bluetooth employs variably sized packets up to a maximum of five frequency-hopping time-slots of 625 microseconds in duration. In Bluetooth’s Time Division Duplex (TDD) system for transmit/receive separation every frame consists of a packet transmitted from a transmitter node over 1, 3, or 5 timeslots, while a receiver replies with a packet occupying at least one slot, with the result that each frame has an even number of slots. Bluetooth’s v. 2.1 Enhanced Data Rate (EDR) [26] supports gross air rates of 2.0?Mbps (maximum gross user payload rate of 1.4485?Mbps) and 3.0?Mbps (2.1781?Mbps) through respectively, /4-Differential Quadrature Phase-Shift Keying (DQPSK) or 8DPSK modulation. In EDR, the symbol rate (1?Msps) remains the same as in the basic gross air rate of 1.0?Mbps.
In Bluetooth as it stands, Data Medium (DM) packets transmitted at its basic rate (1?Mbps gross air rate) include payload error coding at rates of 1/3 and 2/3 through Hamming block codes. The ability of block codes to cope with burst errors is considerably improved if some form of data interleaving is implemented, because interleaving tends to remove error correlations, with the extent of interleaving being directly related to the maximum burst length resisted. In [27], Bluetooth data interleaving was indeed applied within a packet to bits within FEC-protected blocks. Unfortunately, in bit-level interleaving as applied in [27], the extent of interleaving is restricted to the packet length. Consequently, this form of interleaving only provides burst length resistance to the width of the block reordering matrix.
Because of the variety of Bluetooth packet types and the additional modulation modes, it is important in any experiments or simulations to establish which packet type should be selected for a particular channel Carrier-to-Noise Ratio (CNR). Table 1 summarizes all the Asynchronous Connectionless (ACL) mode EDR packet types currently available (according to the specification) as well as EDR DM-type packets (suggested as additional packet types) in the event that bit-level FEC was to be added to EDR. The consequences of adding the bit-level FEC EDR packet types in terms of calculated packet error rates and choice of optimal packet types are reported in [28]. In Table 1, only the maximum forward asymmetric data rates are shown, assuming that a single time-slot return packet is also included in the Bluetooth frame.
Importantly, the Bluetooth packet user payload includes a one or two byte Cyclic Redundancy Check (CRC) to detect packet error with a high likelihood. This means that in packet-level FEC an incorrectly received packet can be detected, and erasure correction becomes appropriate. For RS coding, packet error detection by CRC results in the error location polynomial. The incorrect packet can be discarded, as it can be recreated from correctly received packets with the help of the additional FEC-only packets.
3.2. Packetization of Video
Unfortunately, if packetization takes place on a single video slice (one row of macroblocks in the MPEG-2 codec equivalent to a GOB in H.263) per Bluetooth packet the result is partially filled packets as well as many 1- or 3-slot packets, with a consequent drop in throughput (see Figure 4). Therefore, in [29] fully filled Bluetooth packets were formed, regardless of slice boundaries. While this results in some loss in error resilience, as each slice contains a decoder synchronization marker, in [29] it is shown that the overall video performance is superior, as the throughput is superior. Forming constant-sized Bluetooth packets has an important advantage over the packet-level FEC scheme in [30], because it avoids the need to pad unfilled packets and calculate FEC on those zero values. The H.264 codec supports variably sized slices, but, without knowledge in advance of channel conditions, the optimal packet mode and, consequently, the optimal slice size are not known, and so in that respect H.264 is in no better position than earlier codecs with a limit on maximum slice extent.
3.3. Channel Model
A two-state discrete-time, ergodic Markov chain models the wireless channel error characteristics between a Bluetooth master and slave node. To simplify the interpretation probability of bit error in the good state was set at 0, while the probability of bit error in the bad state was set to 1. A similar channel model was adopted for the very same reason in [13]. However, in that research the time in any state was quantized in proportion to the frame duration, whereas, in our experiments, after each bit-duration decisions are taken. In the experiments without extension to priority encoding, the mean duration of a good state was set to 30?kb or equivalently with 2-DH5 type packets (Table 1) a duration of 5.5 packets, which corresponds to a probability of going from a good state (g) to a bad state (b) of as For the extended priority encoding FEC interleaving scheme, the mean duration of the good state was set to 108.64?kb or equivalently to 20 packets (2-DH5 type). This results in . The longer mean good state duration in the extended scheme tests actually results in a wider variety of state durations, because state decisions are made every bit-duration.
We were pleased to employ the University of Cincinatti Bluetooth (UCBT) extension to the well-known ns-2 network simulator (v. 2.28 used). The UCBT extension supports Bluetooth EDR but is also built on the air models of previous Bluetooth extensions such as BlueHoc from IBM and Blueware.
4. Results
Simulations were carried out with input from an MPEG-2-encoded bitstream at a mean rate of 700?kbit/s for a 40-second video clip with moderate motion, showing a newsreader and changing backdrop, which we designate “News”. In some experiments by way of comparison, we used a clip from “Friends” from the well-known American situational comedy, with more “action” than in “News”, but otherwise with the same encoding characteristics. The frame rate for both clips was 25?Hz in each run resulting in 1000 frames in all. The source video was European Source Input Format (SIF) (352 × 288 pixel/frame) with a GOP structure of and . Error concealment was by previous frame replacement, which is a simple but standard scheme [3]. Luminance PSNR was found by reconstructing with a reference MPEG-2 decoder. The Digital Video Broadcasting consortium and other important providers continue to employ MPEG-2 for digital multimedia services. The majority of legacy video also remains in MPEG-2 format. Research is ongoing to achieve efficient transcoding between MPEG-2 and the H.264/Advanced Video Codec (AVC) [31], as this can be a very time-consuming process. In tests on the Tempete clip (containing camera zoom; spatial detail; fast random motion, that is, with high coding complexity) [32], at about 700?k bps, the MPEG-2 was still able to achieve approximately 30?dB (PSNR), which is generally regarded as good quality for mobile users, whereas at the same rate H.26L (the prototype for H.264), the quality was about 34?dB. However, there is an increase in H.264 decoder complexity of 3.5 times for Main profile (necessary to create B-pictures, as in our tests) or 2.5 times for Baseline profile. This complexity increase must be balanced against the need to preserve battery longevity on mobile devices. The broad principles of interleaving apply whichever codec is used.
4.1. FEC Interleaving without Priority Encoding
Figure 6 plots values of selected by the algorithm against the display deadline setting, assuming RS(3,2), that is, two video packets for every one FEC packet. The relationship is almost linear, confirming the required behavior of the algorithm. Notice that the value of was artificially set in the tests but would in the long term remain stable for a fixed input video rate. In those circumstances, the value of is largely determined by the size of the playout buffer at the receiver device.
Figures 7 and 8 are a comparison of sub-packet-level interleaving and packet interleaving for , 8. The figures show packet loss and matching video quality, after averaging over 20 runs to establish convergence. The packet size was for 2-DM5 type (Table 1), and the channel model was the simplified Markovian burst error channel of Section 3.3. For symbol interleaving within a packet, the block size was 15 bits, and the expurgated (15, 10) Hamming code was employed with double consecutive error correction. The packet-level FEC ratio for RS erasure correction coding was set at the same level as for the Bluetooth Hamming code, that is, a rate of 2/3. This means that for every two packets one FEC packet was generated.
It is clear from Figures 7 and 8 that there is a significant advantage from packet interleaving and that advantage grows if the level of interleaving increases from a maximum of four packets to a maximum of eight packets (not including FEC packets). The only situation in which symbol interleaving within a packet is competitive is when packet burst size is short. This is not surprising, as sub-packet-level interleaving only converts burst errors to random errors for burst sizes less than bits. However, the degree of difference is surprising, given that the PSNR plots are on a logarithmic scale. Applying FEC without interleaving results in the lowest delivered video quality.
4.2. FEC Interleaving with Priority Encoding
The extended FEC interleaving scheme with priority encoding leads to physical interleaving of the packet transmission order. As the scheme involves forming potentially irregularly sized FEC subgroups, it is not immediately clear what the effect of this policy is. Figure 9 shows that, for the “News” clip, there is almost a linear increase in the average packet distance between formerly adjacent packets, with increase to global FEC group size . The number of successfully recovered packets also increases when the global FEC group size is increased, Figure 10, and is distinctly better than the built-in Bluetooth FEC scheme (but applied to 2-DM5 packets), marked as “no interleaving” for comparison with Figures 7 and 8. However, notice that the channel model in Figure 8 is that for the extended scheme, as described in Section 3.3.
The improvement afforded by priority encoding is found in Figure 11 when streaming the “News” clip across the Bluetooth channel. The mean burst length was progressively increased, while both FEC interleaving schemes were tried (mean of ten simulation runs). For I-picture packets under the priority-encoded scheme, Figure 11(a), the packet loss ratio (number of lost packets to total number of packets transmitted) is reduced when priority-encoded FEC interleaving is applied. The advantage of priority-encoded FEC interleaving improves with worsening channel condition. Similar remarks apply to P-picture packets in Figure 11(b), though the advantage of the priority encoding scheme is not so noticeable. In Figure 11(c) the ratio is reversed, which is as expected. For a fixed redundancy allowance, then if I- and P- picture packets are given enhanced protection, then this can only come at the cost of less protection for B-picture packets.
(a)
(b)
(c)
Figure 12 compares FEC interleaving schemes with and without priority encoding in terms of received video quality, for . The priority-encoded FEC interleaving scheme (marked Unequal Priority (UP) in the plots) improves the luminance PSNR according to the depth of the error burst and generally with the complexity of the compressed video stream. For large bursts during transfer of a video clip with moderate motion the quality drops below 30?dB. For small bursts either scheme is able to maintain good quality at the 40?dB level. With set to 12, the quality is best. Even with set to 6, the priority-encoded FEC interleaving scheme outperforms the others, except at low error rates, when it results in an equivalent quality.
In Figure 12, when the Pro-MPEG Cop#3r2 [10] interleaving scheme is applied with equivalent FEC protection, but without priority encoding, it is approximately similar in behaviour as far as video quality is concerned to the interleaving scheme in this paper without priority encoding. However, its performance is notably below that of the scheme with priority encoding as the mean burst packet burst length increases. The point that the superiority becomes apparent depends on the video coding complexity. We have also plotted the performance of the Pro-MPEG Cop#3r2 [10] interleaving scheme without the overhead of FEC. It will be apparent that without FEC protection the video quality quickly becomes unacceptable as the packet error rate increases. Therefore, the bandwidth savings arising from using this scheme without FEC cannot be tolerated in this type of wireless environment.
In Figure 13, selected results from Figure 12 for packet type 2DH-5 have been repeated but with packet size 3DH-5. In Figure 13, the increase in FEC protection reduces errors but the increase in packet length increases the risk of error compared to a packet length of type 2DH-5 (refer to Table 1). The net result is a decrease in overall video quality between Figures 12 and 13. Nevertheless, the proposed priority-encoded FEC interleaving scheme results in relative higher quality video compared to the Pro-MPEG Cop#3r2 interleaving scheme. Notice in Figure 13 that the burst length scale is the same as in Figure 12, that is, in units of 2DH-5 packets, in order to allow comparison.
5. Conclusions
Subpacket-level interleaving with FEC is commonly applied to combat correlated error bursts across a wireless channel. However, this paper has shown that packet-level interleaving, when used with RS coding, in the Bluetooth case study, results in better video quality compared to sub-packet-level interleaving. Given that Bluetooth already detects packet error through a CRC, packet-level interleaving with FEC is preferable. However, interleaving causes delay. Therefore, this paper also introduces a delay-aware algorithm for calculating the extent of interleaving. The algorithm is conservative, avoiding any risk of missed delays by taking into account the position of packets within an interleaving matrix and the picture type importance of the packets. Moreover, because estimates of packet arrival rate are required, the algorithm decides the extent of interleaving at the last moment. When the error burst extends over several packets during a deep fade, then priority-encoded FEC interleaving becomes attractive despite its added complexity. To engineer the solution, the number of FEC packets was fixed while the number of information packets was varied. This allowed a greedy algorithm to be applied. The algorithm is suboptimal but is not over complex. Both packet-level FEC interleaving schemes (with or without priority encoding) produce good quality delivered video when bursts extend over two packets. In harsher conditions, a significant improvement is provided by extending to priority encoding. When priority encoding is applied, there is a gain in delivered video quality and in reduced FEC overhead compared to the standard broadcasting industry packet interleaving scheme for wireline networks.
Acknowledgment
This work was supported by the EPSRC, UK, under Grant no. EP/C538692/1.