High-Performance Region-of-Interest Image Error Concealment with Hiding Technique

Abstract

Recently region-of-interest (ROI) based image coding is a popular topic. Since ROI area contains much more important information for an image, it must be prevented from error decoding while suffering from channel lost or unexpected attack. This paper presents an efficient error concealment method to recover ROI information with a hiding technique. Based on the progressive transformation, the low-frequency components of ROI are encoded to disperse its information into the high-frequency bank of original image. The capability of protection is carried out with extracting the ROI coefficients from the damaged image without increasing extra information. Simulation results show that the proposed method can efficiently reconstruct the ROI image when ROI bit-stream occurs errors, and the measurement of PSNR result outperforms the conventional error concealment techniques by 2 to 5 dB.

1. Introduction

Advances information systems and networked databases continue to rapid growth in digital media, such as image, audio, and video. The object-based encoder is developed to increase the coding flexibility [1]. Each object is independently processed and transmitted in the multimedia system. The bit-stream may be missed or wrong due to the package lost or channel error, thus the object cannot be normally decoded. In the recent years, error concealment techniques are widely discussed for recovering image or video data [2–9]. For continuous video signals, the motion compensation can be efficiently used to recover the damaged blocks from the interframes [2, 3]. However, as for a still image, the temporal information cannot be used to rescue the error blocks. Instead, only the spatial interpolation is employed to reconstruct the lost pixels for error blocks with adjacent available blocks [4–6]. However, the interpolated result is not preferred since the significant data is hardly interpolated from the adjacent blocks, particularly large-region error happened. To improve the quality of error concealment, the spatial-temporal mixing method is adopted [7–10]. Recently, the watermark-based approach for error detection and recovery is proposed to improve the performance of error concealment [11–13]. Basically, these methods embed the watermarking information to the quantized frequency domain, such as discrete cosine transform (DCT). The decoder can detect the error flag to check whether the received data is corrupted. Then the error concealment technique can recover the corrupted pixel to improve the image quality [13].

Generally, some regions of an image involve much more important information. The significant image can be selected as a ROI region. ROI-based coding had been presented in some documents [14–16]. For example, the JPEG-2000 standard, used ROI coding concept [17]. The receiver can obtain high quality ROI image even if the transmitted bit-stream suffers from channel errors. Although JPEG coding had widely been used in current, the coded image is easily damaged since the original JPEG standard not considers the error-recovery problem [18]. To improve this drawback, the JPEG-2000 can perform high robustness in an error-prone channel. However, it is not appreciated for low-power portable digital camera systems (DCSs) due to high complexity and large memory required. Alternatively, the conventional JPEG coder also can use ROI coding technique to improve its robustness.

This study aims to develop an efficient error concealment algorithm for a still image, specially for ROI area recovering. To achieve better performance, the new algorithm employs an image-in-image technique rather than the interpolation approach. The ROI image can be selected and encoded based on subband filtering. The coefficients of ROI image can be embedded into the original image. When the ROI data cannot be correctly decoded, one can extract the coefficients of the hidden subimage to recover ROI pixels. This paper is organized as follows. The ROI image encoder for error concealment is presented in Section 2. The high robustness for ROI image decoding is described in Section 3. Simulated and comparisons are discussed in Section 4. The concluding remarks are outlined in Section 5.

2. Proposed Error Concealment Method for ROI Image Encoding

The proposed error concealment techniques focus on the ROI image processing. First, we select the ROI from the part of an original image. The ROI image is divided into four subbands, LL band, LH band, HL band, and HH band that represent its 2D frequency components. In order to reconstruct the image without frequency aliasing, the transformation should have the Quadrature Mirror Filter (QMF) feature [19]. The subband filter adapts the discrete cosine transform (DCT) for its wide use. To confirm QMF in DCT, horizontal symmetry, vertical symmetry, and cross symmetry properties must be satisfied. In discrete signal processing, a 2D spatial block {𝑓_𝑗𝑘} is transformed into a DCT block by

𝐹_𝑢𝑣=𝐶_𝑢𝑣^𝑁−1_𝑗=0^𝑁−1_𝑘=0𝑓_𝑗𝑘𝑊^𝑢𝑣_𝑗𝑘=FDCT(𝑓_𝑗𝑘𝑊),_{𝑢𝑣𝑗𝑘}=cos((2𝑗+1)𝑢𝜋2𝑁)cos((2𝑘+1)𝑣𝜋2𝑁), (1) where the symbols (𝑢,𝑣) and (𝑗,𝑘) are the coordinate of the frequency and spatial pixel respectively, 𝐶_𝑢𝑣 is a constant and 𝑁 is the block size. DCT can easily achieve horizontal symmetry, vertical symmetry, and cross symmetry by observing 𝑊^𝑢𝑣_{𝑗(𝑁−𝑘−1)}=(−1)^𝑣𝑊^𝑢𝑣_𝑗𝑘, 𝑊^𝑢𝑣_{(𝑁−𝑗−1)𝑘}=(−1)^𝑢𝑊^𝑢𝑣_𝑗𝑘, and 𝑊^𝑢𝑣_{(𝑁−𝑗−1)(𝑁−𝑘−1)}=(−1)^𝑢+𝑣𝑊^𝑢𝑣_𝑗𝑘, respectively. For 2D subband DCT, we can define each band as follows: (a) LL band 𝐹_LL=𝐹_𝑢1,𝑣1,𝑁/2>|𝑢1|≥0 and 𝑁/2>|𝑣1|≥0; (b) LH band 𝐹_LH=𝐹_𝑢2,𝑣2,𝑁/2>|𝑢2|≥0, and (𝑁−1)>|𝑣2|≥𝑁/2; (c) HL band 𝐹_HL=𝐹_𝑢3,𝑣3,(𝑁−1)>|𝑢3|≥𝑁/2, and 𝑁/2>|𝑣3|≥0; (d) HH band 𝐹_HH=𝐹_𝑢4,𝑣4,(𝑁−1)>|𝑢4|≥𝑁/2, and (𝑁−1)>|𝑣4|≥𝑁/2.

2.1. The ROI Image Embedding Method

First, the ROI image is selected from an image and it is coded with the subband approach. An image-in-image technique is employed to improve the robustness for ROI data. The key components of the ROI image are further embedded to the original image. The original image quality should be kept after ROI data embedding. Let 𝑓_𝑗𝑘 denote a composite image where a ROI image 𝑅_𝑗𝑘 has been embedded into an original image 𝑓_𝑗𝑘. To avoid the visible distortion, the composite image must be very close to the original image. Intuitively, the high frequency portion of the original image is an ideal place to insert ROI information since the great variation in high frequency makes much more difficult to detect in vision. As the original image is divided into 𝑁×𝑁 blocks, its subbands can be obtained from

Figure 1 

The original image transformation with block DCT.

Figure 2 

The ROI image transformation with full-size DCT.

𝐹_𝑢𝑣=FDCT(𝑓_𝑗𝑘)_𝑁×𝑁=𝐹_LL𝐹_LH𝐹_HL𝐹_HH𝑁×𝑁=𝐹_𝑢1𝑣1𝐹_𝑢2𝑣2𝐹_𝑢3𝑣3𝐹_{𝑢4𝑣4𝑁×𝑁}. (2) The illustration is shown in Figure 1, for 2D block transformation.

In order to disperse the spatial correlation, the ROI image 𝑅_𝑗𝑘 is also transformed by DCT with full picture. Then the DCT coefficients can be split into subbands as

𝑅_𝑢𝑣=FDCT(𝑅_𝑗𝑘)_𝑚1×𝑚2=𝑅_LL𝑅_LH𝑅_HL𝑅_{HH𝑚1×𝑚2}, (3) where the ROI image size used 𝑚1×𝑚2. The illustration is shown in Figure 2, for full-picture transformation. Since the 𝑅_LL band contains more information about the ROI image, only their DCT coefficients are embedded into the high-frequency band of the original block in (2). Due to full-picture transformation, the value of the LL band coefficient is usually large. These coefficients can be quantized before the embedding operation, to reduce visual distortion. Then (3) is further simplified to

𝑅_𝑢𝑣=1𝑄_𝑚1𝑚2𝑅_LL000_𝑚1×𝑚2=𝑅_LL000_𝑚1×𝑚2, (4) where 𝑄_𝑚1𝑚2 are parameters from a quantization table. Note that the rank of 𝑅_LL is one quarter of 𝑅_𝑢𝑣. In order to save the bit-rate of the ROI image further, one can repeat the above procedure to produce a second layer LL band 𝑅_LL from the first layer 𝑅_LL. Thus total bit-rate of the ROI image is now reduced to 1/16 of the original one.

For image-in-image processing, the coefficient of 𝑅_LL is sequentially embedded into the highest frequency of 𝐹_HH band of the original DCT block. To achieve good hiding quality and strong robustness, the embedding procedures are proposed with the following two cases:

(i)if 𝐹(𝑚,𝑛)_max and 𝐶_𝑢𝑣 are the same sign, 𝐹(𝑚,𝑛)_max=𝐶_𝑢𝑣,|𝐶_𝑢𝑣𝐶|iseven,_𝑢𝑣−sign(𝐶_𝑢𝑣),|𝐶_𝑢𝑣|isodd,sign(𝑥)=1,𝑥≥0,−1,𝑥<0, (5) where 𝐹(𝑚,𝑛)_max is the highest frequency component at the (𝑚,𝑛)th block and 𝐶_𝑢𝑣 is the one of embedded coefficient at the (𝑢,𝑣) position in 𝑅_LL matrix. With (5), the embedded coefficient can be forced to even,(ii)if 𝐹(𝑚,𝑛)_max and 𝐶_𝑢𝑣 are the different sign, 𝐹(𝑚,𝑛)_max=−(𝐶_𝑢𝑣−sign(𝐶_𝑢𝑣)),|𝐶_𝑢𝑣|iseven,−𝐶_𝑢𝑣,|𝐶_𝑢𝑣|isodd. (6) In this case, one can force the embedded coefficient to odd. The example is shown in Tables 1 and 2. At the decoder, we can change the coefficient sign according to the 𝐹(𝑚,𝑛)_max that is even or odd. The detail is described in the next section. With this approach, the coefficient sign is not changed after embedding procedure, and so the quality does not degrade seriously. After embedding, the transformed block becomes 𝐹_𝑢𝑣=𝐹_LL𝐹_LH𝐹_HL𝐹_HH𝑁×𝑁, (7) where 𝐹_HH contains one coefficient for ROI image. However, if the value of the embedded ROI coefficient is too large, the image will appear visible distortion after embedding. To avoid this drawback, the maximum value 𝑅_max of the quantized ROI coefficient is limited.

Table 1 

Example for 𝑓(𝑚,𝑛)_max and 𝐶_𝑢𝑣 with the same sign.

Table 2 

Example for 𝑓(𝑚,𝑛)_max and 𝐶_𝑢𝑣 with the different sign.

Finally, DCT coefficients are converted into spatial domain by using the inverse DCT transformation, hence the composite image is obtained by 𝑓_𝑗𝑘=IDCT(𝐹_𝑢𝑣) of each subblock. Using this embedding procedure, we do not require increasing bit-rate for ROI image hiding, but sacrifice the original image quality a little. The complete embedding procedure of our algorithm is illustrated in Figure 3. In summary, the embedding procedures are the following.

Figure 3 

The encoding procedure for ROI image hiding procedure.

(1)Select ROI image from the original image.(2)Transform the ROI image and the original image to DCT domain.(3)Pick up LL band of ROI image, and then quantize and down-sample the coefficients. (4)Embed the quantized LL-band coefficient to the HH-band of the original block sequentially.(5)Inversely transform the DCT coefficients of original blocks to the spatial pixels.

It is worthy notice that we need to take a tradeoff between quality and robustness. While ROI robustness is enhanced, the quality of original image would be degraded since more ROI coefficients are embedded. To keep the balance between quality and robustness, only one ROI coefficient is embedded to one DCT block of original image. To avoid degrading the quality of the original image, the block size uses 16×16 in (2). The quality degrades very slightly due to only 1/256 coefficient changed. If the original image size is M×N, so there are (M/16×N/16) DCT blocks. The ROI image size is selected from 1/16 of original image. So ROI subimage has M×N/16 pixels. The embedded coefficients only select from 1/16 of original ROI coefficients. Hence we can embed M×N/(16×16) coefficients to (M/16×N/16) DCT blocks. This can achieve good tradeoff between quality and robustness. In the embedding scheme, one DCT coefficient is changed in the 16×16 block, but keeping the coefficient sign. Hence the quality of composite image could be very close to the original one. Besides, high-robustness ROI image can be provided in the error-prone channel.

3. Error Concealment Decoder for ROI Image

Generally, the decoding flow is an inverse operation of encoder. Figure 4 shows the decoding flow for extracting and recovering ROI content when the ROI data is lost. First the relative coordinate of ROI region is obtained from the encoder. When the decoder receives the composite image from the communication channel, one can check whether ROI data is correct from the forward error correction (FEC) module of the channel coding [20]. If no FEC, we also can automatically check ROI pixels lost by decoding data and slice start code to keep the synchronization between encoder and decoder [21]. While the ROI pixels are corrupted, the received image is transformed to DCT domain. Then one coefficient of ROI image can be extracted from HH band of the correct block. The ROI coefficient can be restored from

Figure 4 

The decoding flow for ROI image extraction.

𝑅_𝑢𝑣=𝐹(𝑚,𝑛)_max,|𝐹(𝑚,𝑛)_max|iseven,−𝐹(𝑚,𝑛)_max|𝐹(𝑚,𝑛)_max|isodd. (8) If 𝐹(𝑚,𝑛)_max is even, the 𝐹(𝑚,𝑛)_max can be directly extracted as the ROI coefficient since the sign of ROI coefficient is the same as 𝐹(𝑚,𝑛)_max. On the other hand, we need to change the sign to avoid producing the serious distortion. The ROI coefficients can be sequentially extracted from the HH band of each block. Finally, the LL band matrix of the ROI image can be approximately reconstructed. To restore the ROI image, one can combine the LH band, HL band and HH band of the ROI image for a complete 2D subband processing. However, data in these bands is not used, so zeros are inserted to LH, HL and HH bands to achieve a complete matrix. After a dequantization procedure, the coefficient matrix 𝑅_𝑢𝑣 of the ROI image can be reconstructed. Finally the ROI image can be decoded from the inverse DCT, and then the restored ROI pixels can be pasted to the damaged region. In summary, ROI image error concealment technique follows.

(1)Transform the receiving image to DCT domain.(2)Exact the coefficients of ROI image from HH band of each block sequentially.(3)Restore the LL-band coefficient of ROI image.(4)Pad zeros to LH, HL and HH bands of ROI image.(5)Inverse DCT transforms the coefficients to spatial image.(6)Paste the extracted ROI pixels to the original image if ROI region is damaged.

The proposed decoding method first extracts the ROI coefficients from the correct blocks of the composite image. Due to the fact that DCT coefficients have the progressive feature, one coefficient represents one spatial resolution. The resolution of the image shall become better and better as the correct coefficients are successively accumulated into the transformed kernel. For the case where the composite image is damaged in transmission, only the part of the coefficients is lost in subbands for ROI image recovering. ROI image can be approximately reconstructed from coefficients of other intact blocks. Then one can extract ROI image from the damaged image itself to recover the ROI pixel.

4. Simulations and Comparisons

In the experiment, the original image employed 512×512 pixels with 8-bit resolutions. The block size used 16×16 for DCT transform in (1). So there are (512/16)× (512/16) = 1024 blocks that can be embedded by 1024 coefficients of ROI image. The ROI image is selected with 128×128 block size of the original image. With two-layer subband down sampling, we can embed 128×128/16=1024 coefficients to the original DCT blocks. The number of original blocks is exactly equal to the number of embedded coefficients of ROI image, to achieve a good tradeoff between quality and robustness. The maximum size for ROI image could be selected with 1/16 of the original image to meet our expectation. To avoid the embedding distortion, the value of the maximum ROI coefficient is limited by taking 𝑅_max=100∼200, where high 𝑅_max is used as near to the DC coefficient. Then the ROI coefficients are sequentially embedded to the highest frequency position of the HH-band of each block with (5) and (6).

The test used eight standard pictures as benchmarks, as shown in Figure 5. One can select ROI image from the original image. First, various conditions are estimated with the embedding block size 16×16, and 8×8, respectively, and the embedding position select the middle and high frequency bands, respectively. The results are shown in Figure 6. Clearly, the best embedding quality is to insert the data to high-frequency band with 16×16 block. Next, one can employ various quantization step (Q) in (4) to evaluate the embedding quality of each picture. The results are shown in Figure 7. When 𝑄>30, the embedding quality can be stable. When Q increases, the quality of ROI image extracted would be poor. Clearly, 𝑄=30 can be selected as the best point, to keep both of the quality of original image and extracted ROI image. Figure 8 shows the quality of ROI image extracted under various Q values. Results show that 𝑄=30 can satisfy the quality of error concealment. Figure 9 shows the embedding results for each testing picture. The quality of composite image is very good since PSNR value is close to 50 dB. The hiding process degrades the original image quality very slightly. Hence the user cannot detect any difference between the original image and composite image in vision. Figure 9 shows, the quality of error concealment when ROI image is damaged. The error ROI pixels can be efficiently restored. This approach not only can keep the quality of original image but also achieve superb quality for ROI image recovery.

Figure 5 

The original image and its selected ROI image.

Figure 6 

The ROI image embedded into an original image in various conditions.

Figure 7 

The quality of composite image as ROI embedded in various quantization.

Figure 8 

The quality of error-recovery image as ROI extracted in various quantization.

Figure 9 

The result of ROI image embedded into an original image

Figure 10 

The error recovery with ROI image extraction.

Next, we compare the other error-concealment techniques and the proposed method. Chen et al. [13] present the fragile watermark that embeds watermarking information to high-frequency components using Force-Even-Watermark (FEW) approach. At the decoder, we can check whether the block is error from its odd coefficient. For a still image, the spatial interpolation can be used to recover the error block. Table 3 lists the PSNR results with various methods. The simple bilinear interpolation method can improve the quality for error images, but the performance is not preferred due to lack of spatial correlation. The edge-based interpolation methods [4, 5] were presented to promote the interpolation quality. However, the significant edge is very difficult to find as large ROI pixels are missed. This method can improve about 1∼2 dB PSNR compared with bilinear method. The performance is still not good since the correlation between ROI image and the adjacent data is quite low. The proposed algorithms with the hiding scheme can greatly improve the ROI image quality. For video/image communication, the forward-error correction (FEC) is always used. If FEC cannot recover the error bit, the error flag will become high. At the receiver, one can check the error flag whether it is high. When the error flag is high in ROI region, the hiding ROI coefficient can be restored to recover the error data. Based on progressive DCT coefficients, one can also get enough ROI image quality from the coefficients of the intact blocks. Results shown that detail ROI content can be efficiently restored and its PSNR can be efficiently improved.

Table 3 

Comparisons with other error concealment algorithms for ROI image recovery.

For a real case, the image may be corrupted both in ROI image and non-ROI image. To evaluate the robustness performance, we randomly destroy the transmitted image, where included ROI and non-ROI errors. The various error rates are tested, and the results are listed in Table 4. The corrupted image included 6.25% ROI image and other ratio non-ROI image. For example, when the error ratio of a corrupted image is 20%, the ROI image and non-ROI image are 6.26% and 13.74% respectively. The ROI image is recovered with the proposed hiding method, and the non-ROI image is recovered with our previous spatial interpolation algorithm [4]. When the non-ROI image is also destroyed, we also can extract the ROI coefficients from the residual intact blocks to rescue the ROI pixel. The restored ROI image quality depends on the number of correct blocks. When the number of error block increases, the recovery performance degrades since the correct coefficient reducing accordingly.

Table 4 

PSNR computations for ROI image recovery in various error ratio.

5. Conclusions

The error concealment algorithm for large ROI region of still image is not easily surmounted. In this study, the novel error concealment algorithm is presented with image hiding approach. Based on subband DCT theory, the LL band of ROI image is dispersed to be embedded to the HH band of the original image. At the receiver, this approach can recover the damaged images itself without adding the extra information. After embedding, the original image quality degrades very slightly, and the error robustness for ROI image can be significantly promoted. The proposed method can efficiently recover the detailed ROI content and the PSNR quality can improve about 2∼5 dB with respect to the conventional spatial interpolation algorithms.

Acknowledgment

This work was supported by the National Science Council, Taiwan under NSC92-2213-E-327-010.