Research Article | Open Access
High-Performance Region-of-Interest Image Error Concealment with Hiding Technique
Recently region-of-interest (ROI) based image coding is a popular topic. Since ROI area contains much more important information for an image, it must be prevented from error decoding while suffering from channel lost or unexpected attack. This paper presents an efficient error concealment method to recover ROI information with a hiding technique. Based on the progressive transformation, the low-frequency components of ROI are encoded to disperse its information into the high-frequency bank of original image. The capability of protection is carried out with extracting the ROI coefficients from the damaged image without increasing extra information. Simulation results show that the proposed method can efficiently reconstruct the ROI image when ROI bit-stream occurs errors, and the measurement of PSNR result outperforms the conventional error concealment techniques by 2 to 5 dB.
Advances information systems and networked databases continue to rapid growth in digital media, such as image, audio, and video. The object-based encoder is developed to increase the coding flexibility . Each object is independently processed and transmitted in the multimedia system. The bit-stream may be missed or wrong due to the package lost or channel error, thus the object cannot be normally decoded. In the recent years, error concealment techniques are widely discussed for recovering image or video data [2–9]. For continuous video signals, the motion compensation can be efficiently used to recover the damaged blocks from the interframes [2, 3]. However, as for a still image, the temporal information cannot be used to rescue the error blocks. Instead, only the spatial interpolation is employed to reconstruct the lost pixels for error blocks with adjacent available blocks [4–6]. However, the interpolated result is not preferred since the significant data is hardly interpolated from the adjacent blocks, particularly large-region error happened. To improve the quality of error concealment, the spatial-temporal mixing method is adopted [7–10]. Recently, the watermark-based approach for error detection and recovery is proposed to improve the performance of error concealment [11–13]. Basically, these methods embed the watermarking information to the quantized frequency domain, such as discrete cosine transform (DCT). The decoder can detect the error flag to check whether the received data is corrupted. Then the error concealment technique can recover the corrupted pixel to improve the image quality .
Generally, some regions of an image involve much more important information. The significant image can be selected as a ROI region. ROI-based coding had been presented in some documents [14–16]. For example, the JPEG-2000 standard, used ROI coding concept . The receiver can obtain high quality ROI image even if the transmitted bit-stream suffers from channel errors. Although JPEG coding had widely been used in current, the coded image is easily damaged since the original JPEG standard not considers the error-recovery problem . To improve this drawback, the JPEG-2000 can perform high robustness in an error-prone channel. However, it is not appreciated for low-power portable digital camera systems (DCSs) due to high complexity and large memory required. Alternatively, the conventional JPEG coder also can use ROI coding technique to improve its robustness.
This study aims to develop an efficient error concealment algorithm for a still image, specially for ROI area recovering. To achieve better performance, the new algorithm employs an image-in-image technique rather than the interpolation approach. The ROI image can be selected and encoded based on subband filtering. The coefficients of ROI image can be embedded into the original image. When the ROI data cannot be correctly decoded, one can extract the coefficients of the hidden subimage to recover ROI pixels. This paper is organized as follows. The ROI image encoder for error concealment is presented in Section 2. The high robustness for ROI image decoding is described in Section 3. Simulated and comparisons are discussed in Section 4. The concluding remarks are outlined in Section 5.
2. Proposed Error Concealment Method for ROI Image Encoding
The proposed error concealment techniques focus on the ROI image processing. First, we select the ROI from the part of an original image. The ROI image is divided into four subbands, LL band, LH band, HL band, and HH band that represent its 2D frequency components. In order to reconstruct the image without frequency aliasing, the transformation should have the Quadrature Mirror Filter (QMF) feature . The subband filter adapts the discrete cosine transform (DCT) for its wide use. To confirm QMF in DCT, horizontal symmetry, vertical symmetry, and cross symmetry properties must be satisfied. In discrete signal processing, a 2D spatial block is transformed into a DCT block by
where the symbols and are the coordinate of the frequency and spatial pixel respectively, is a constant and is the block size. DCT can easily achieve horizontal symmetry, vertical symmetry, and cross symmetry by observing , , and , respectively. For 2D subband DCT, we can define each band as follows: (a) LL band and (b) LH band , and (c) HL band , and (d) HH band , and
2.1. The ROI Image Embedding Method
First, the ROI image is selected from an image and it is coded with the subband approach. An image-in-image technique is employed to improve the robustness for ROI data. The key components of the ROI image are further embedded to the original image. The original image quality should be kept after ROI data embedding. Let denote a composite image where a ROI image has been embedded into an original image To avoid the visible distortion, the composite image must be very close to the original image. Intuitively, the high frequency portion of the original image is an ideal place to insert ROI information since the great variation in high frequency makes much more difficult to detect in vision. As the original image is divided into blocks, its subbands can be obtained from
The illustration is shown in Figure 1, for 2D block transformation.
In order to disperse the spatial correlation, the ROI image is also transformed by DCT with full picture. Then the DCT coefficients can be split into subbands as
where the ROI image size used The illustration is shown in Figure 2, for full-picture transformation. Since the band contains more information about the ROI image, only their DCT coefficients are embedded into the high-frequency band of the original block in (2). Due to full-picture transformation, the value of the LL band coefficient is usually large. These coefficients can be quantized before the embedding operation, to reduce visual distortion. Then (3) is further simplified to
where are parameters from a quantization table. Note that the rank of is one quarter of . In order to save the bit-rate of the ROI image further, one can repeat the above procedure to produce a second layer LL band from the first layer . Thus total bit-rate of the ROI image is now reduced to 1/16 of the original one.
For image-in-image processing, the coefficient of is sequentially embedded into the highest frequency of band of the original DCT block. To achieve good hiding quality and strong robustness, the embedding procedures are proposed with the following two cases:if and are the same sign, where is the highest frequency component at the th block and is the one of embedded coefficient at the position in matrix. With (5), the embedded coefficient can be forced to even,if and are the different sign, In this case, one can force the embedded coefficient to odd. The example is shown in Tables 1 and 2. At the decoder, we can change the coefficient sign according to the that is even or odd. The detail is described in the next section. With this approach, the coefficient sign is not changed after embedding procedure, and so the quality does not degrade seriously. After embedding, the transformed block becomes where contains one coefficient for ROI image. However, if the value of the embedded ROI coefficient is too large, the image will appear visible distortion after embedding. To avoid this drawback, the maximum value of the quantized ROI coefficient is limited.
Finally, DCT coefficients are converted into spatial domain by using the inverse DCT transformation, hence the composite image is obtained by ) of each subblock. Using this embedding procedure, we do not require increasing bit-rate for ROI image hiding, but sacrifice the original image quality a little. The complete embedding procedure of our algorithm is illustrated in Figure 3. In summary, the embedding procedures are the following.
It is worthy notice that we need to take a tradeoff between quality and robustness. While ROI robustness is enhanced, the quality of original image would be degraded since more ROI coefficients are embedded. To keep the balance between quality and robustness, only one ROI coefficient is embedded to one DCT block of original image. To avoid degrading the quality of the original image, the block size uses in (2). The quality degrades very slightly due to only 1/256 coefficient changed. If the original image size is , so there are (M/1/16) DCT blocks. The ROI image size is selected from 1/16 of original image. So ROI subimage has /16 pixels. The embedded coefficients only select from 1/16 of original ROI coefficients. Hence we can embed /(16) coefficients to (M/1/16) DCT blocks. This can achieve good tradeoff between quality and robustness. In the embedding scheme, one DCT coefficient is changed in the 16 block, but keeping the coefficient sign. Hence the quality of composite image could be very close to the original one. Besides, high-robustness ROI image can be provided in the error-prone channel.
3. Error Concealment Decoder for ROI Image
Generally, the decoding flow is an inverse operation of encoder. Figure 4 shows the decoding flow for extracting and recovering ROI content when the ROI data is lost. First the relative coordinate of ROI region is obtained from the encoder. When the decoder receives the composite image from the communication channel, one can check whether ROI data is correct from the forward error correction (FEC) module of the channel coding . If no FEC, we also can automatically check ROI pixels lost by decoding data and slice start code to keep the synchronization between encoder and decoder . While the ROI pixels are corrupted, the received image is transformed to DCT domain. Then one coefficient of ROI image can be extracted from HH band of the correct block. The ROI coefficient can be restored from
If is even, the can be directly extracted as the ROI coefficient since the sign of ROI coefficient is the same as . On the other hand, we need to change the sign to avoid producing the serious distortion. The ROI coefficients can be sequentially extracted from the HH band of each block. Finally, the LL band matrix of the ROI image can be approximately reconstructed. To restore the ROI image, one can combine the LH band, HL band and HH band of the ROI image for a complete 2D subband processing. However, data in these bands is not used, so zeros are inserted to LH, HL and HH bands to achieve a complete matrix. After a dequantization procedure, the coefficient matrix of the ROI image can be reconstructed. Finally the ROI image can be decoded from the inverse DCT, and then the restored ROI pixels can be pasted to the damaged region. In summary, ROI image error concealment technique follows.Transform the receiving image to DCT domain.Exact the coefficients of ROI image from HH band of each block sequentially.Restore the LL-band coefficient of ROI image.Pad zeros to LH, HL and HH bands of ROI image.Inverse DCT transforms the coefficients to spatial image.Paste the extracted ROI pixels to the original image if ROI region is damaged.
The proposed decoding method first extracts the ROI coefficients from the correct blocks of the composite image. Due to the fact that DCT coefficients have the progressive feature, one coefficient represents one spatial resolution. The resolution of the image shall become better and better as the correct coefficients are successively accumulated into the transformed kernel. For the case where the composite image is damaged in transmission, only the part of the coefficients is lost in subbands for ROI image recovering. ROI image can be approximately reconstructed from coefficients of other intact blocks. Then one can extract ROI image from the damaged image itself to recover the ROI pixel.
4. Simulations and Comparisons
In the experiment, the original image employed 5112 pixels with 8-bit resolutions. The block size used for DCT transform in (1). So there are (512/16) (512/16) = 1024 blocks that can be embedded by 1024 coefficients of ROI image. The ROI image is selected with 1228 block size of the original image. With two-layer subband down sampling, we can embed 12 coefficients to the original DCT blocks. The number of original blocks is exactly equal to the number of embedded coefficients of ROI image, to achieve a good tradeoff between quality and robustness. The maximum size for ROI image could be selected with 1/16 of the original image to meet our expectation. To avoid the embedding distortion, the value of the maximum ROI coefficient is limited by taking where high is used as near to the DC coefficient. Then the ROI coefficients are sequentially embedded to the highest frequency position of the HH-band of each block with (5) and (6).
The test used eight standard pictures as benchmarks, as shown in Figure 5. One can select ROI image from the original image. First, various conditions are estimated with the embedding block size 16, and , respectively, and the embedding position select the middle and high frequency bands, respectively. The results are shown in Figure 6. Clearly, the best embedding quality is to insert the data to high-frequency band with block. Next, one can employ various quantization step (Q) in (4) to evaluate the embedding quality of each picture. The results are shown in Figure 7. When , the embedding quality can be stable. When Q increases, the quality of ROI image extracted would be poor. Clearly, can be selected as the best point, to keep both of the quality of original image and extracted ROI image. Figure 8 shows the quality of ROI image extracted under various Q values. Results show that can satisfy the quality of error concealment. Figure 9 shows the embedding results for each testing picture. The quality of composite image is very good since PSNR value is close to 50 dB. The hiding process degrades the original image quality very slightly. Hence the user cannot detect any difference between the original image and composite image in vision. Figure 9 shows, the quality of error concealment when ROI image is damaged. The error ROI pixels can be efficiently restored. This approach not only can keep the quality of original image but also achieve superb quality for ROI image recovery.
Next, we compare the other error-concealment techniques and the proposed method. Chen et al.  present the fragile watermark that embeds watermarking information to high-frequency components using Force-Even-Watermark (FEW) approach. At the decoder, we can check whether the block is error from its odd coefficient. For a still image, the spatial interpolation can be used to recover the error block. Table 3 lists the PSNR results with various methods. The simple bilinear interpolation method can improve the quality for error images, but the performance is not preferred due to lack of spatial correlation. The edge-based interpolation methods [4, 5] were presented to promote the interpolation quality. However, the significant edge is very difficult to find as large ROI pixels are missed. This method can improve about dB PSNR compared with bilinear method. The performance is still not good since the correlation between ROI image and the adjacent data is quite low. The proposed algorithms with the hiding scheme can greatly improve the ROI image quality. For video/image communication, the forward-error correction (FEC) is always used. If FEC cannot recover the error bit, the error flag will become high. At the receiver, one can check the error flag whether it is high. When the error flag is high in ROI region, the hiding ROI coefficient can be restored to recover the error data. Based on progressive DCT coefficients, one can also get enough ROI image quality from the coefficients of the intact blocks. Results shown that detail ROI content can be efficiently restored and its PSNR can be efficiently improved.
For a real case, the image may be corrupted both in ROI image and non-ROI image. To evaluate the robustness performance, we randomly destroy the transmitted image, where included ROI and non-ROI errors. The various error rates are tested, and the results are listed in Table 4. The corrupted image included 6.25% ROI image and other ratio non-ROI image. For example, when the error ratio of a corrupted image is 20%, the ROI image and non-ROI image are 6.26% and 13.74% respectively. The ROI image is recovered with the proposed hiding method, and the non-ROI image is recovered with our previous spatial interpolation algorithm . When the non-ROI image is also destroyed, we also can extract the ROI coefficients from the residual intact blocks to rescue the ROI pixel. The restored ROI image quality depends on the number of correct blocks. When the number of error block increases, the recovery performance degrades since the correct coefficient reducing accordingly.
|included ROI and (error ratio-) non-ROI image damaged.|
The error concealment algorithm for large ROI region of still image is not easily surmounted. In this study, the novel error concealment algorithm is presented with image hiding approach. Based on subband DCT theory, the LL band of ROI image is dispersed to be embedded to the HH band of the original image. At the receiver, this approach can recover the damaged images itself without adding the extra information. After embedding, the original image quality degrades very slightly, and the error robustness for ROI image can be significantly promoted. The proposed method can efficiently recover the detailed ROI content and the PSNR quality can improve about dB with respect to the conventional spatial interpolation algorithms.
This work was supported by the National Science Council, Taiwan under NSC92-2213-E-327-010.
- “Coding of audio-visual objects: video,” ISO/IECJTC/SC29/WG11, January 1999.
- M.-J. Chen, C.-S. Chen, and M.-C. Chi, “Temporal error concealment algorithm by recursive block-matching principle,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, no. 11, pp. 1385–1393, 2005.
- S. Tsekeridou and I. Pitas, “MPEG-2 error concealment based on block-matching principles,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 10, no. 4, pp. 646–658, 2000.
- S.-C. Hsia, “An edge-oriented spatial interpolation for consecutive block error concealment,” IEEE Signal Processing Letters, vol. 11, no. 6, pp. 577–580, 2004.
- Y. Shi, X. Zhu, J. Xia, and H. Yin, “A fast and efficient spatial error concealment for intra-coded frames,” in Proceedings of the 1st International Congress on Image and Signal Processing (CISP '08), vol. 1, pp. 264–267, Sanya, China, May 2008.
- I. Ismaeil, S. Shirani, F. Kossentini, and R. Ward, “An efficient, similarity-based error concealment method for block-based coded images,” in Proceedings of IEEE International Conference on Image Processing, vol. 3, pp. 388–391, Vancouver, Canada, September 2000.
- T.-H. Wu, G.-L. Wu, C.-Y. Chen, and S.-Y. Chien, “Enhanced temporal error concealment algorithm with edge-sensitive processing order,” in Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS '08), pp. 3466–3469, Seattle, Wash, USA, May 2008.
- M.-H. Jo, H.-N. Kim, and W.-J. Song, “Hybrid error concealments based on block content,” IET Image Processing, vol. 1, no. 2, pp. 141–148, 2007.
- Y. Chen, Y. Hu, O. C. Au, H. Li, and C. W. Chen, “Video error concealment using spatio-temporal boundary matching and partial differential equation,” IEEE Transactions on Multimedia, vol. 10, no. 1, pp. 2–15, 2008.
- S.-C. Hsia, S.-C. Cheng, and S.-W. Chou, “Efficient adaptive error concealment technique for video decoding system,” IEEE Transactions on Multimedia, vol. 7, no. 5, pp. 860–868, 2005.
- I. J. Cox, J. Kilian, F. T. Leighton, and T. Shamoon, “Secure spread spectrum watermarking for multimedia,” IEEE Transactions on Image Processing, vol. 6, no. 12, pp. 1673–1687, 1997.
- C.-T. Hsu and J.-L. Wu, “Hidden digital watermarks in images,” IEEE Transactions on Image Processing, vol. 8, no. 1, pp. 58–68, 1999.
- M. Chen, Y. He, and R. L. Lagendijk, “A fragile watermark error detection scheme for wireless video communications,” IEEE Transactions on Multimedia, vol. 7, no. 2, pp. 201–211, 2005.
- O. T. C. Chen and C. Ch. Chen, “Automatically-determined region of interest in JPEG 2000,” IEEE Transactions on Multimedia, vol. 9, no. 7, pp. 1333–1345, 2007.
- H. Yang, M. Long, and H.-M. Tai, “Region-of-interest image coding based on EBCOT,” IEE Proceedings: Vision, Image and Signal Processing, vol. 152, no. 5, pp. 590–596, 2005.
- Y.-L. Lin, S.-F. Lin, H. H. Chen, and Y.-F. Hsu, “Improving the coding of regions of interest,” in Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS '06), pp. 4313–4316, Kos, Greece, May 2006.
- JPEG2000, ISO/IEC JTC 1/SC29/WG 1 N988, 1998.
- G. K. Wallace, “The JPEG still picture compression standard,” Communications of the ACM, vol. 34, no. 4, pp. 30–44, 1991.
- J. F. Yang, D. Y. Chan, and Y. B. Chen, “Fast and low round-off implementation of quadrature mirror filters for sub-band coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 5, no. 6, pp. 524–523, 1995.
- V. Pless, Introduction to the Theory of Error-Correcting Codes, Wiley, New York, NY, USA, 1998.
- W.-J. Chu and J.-J. Leou, “Detection and concealment of transmission errors in H.261 images,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, no. 1, pp. 74–84, 1998.
Copyright © 2010 Shih-Chang Hsia et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.