Abstract
This paper presents a copyright identification scheme for color images that takes advantage of the complementary nature of watermarking and fingerprinting. It utilizes an authentication logo and the extracted features of the host image to generate a fingerprint, which is then stored in a database and also embedded in the host image to produce a watermarked image. When a dispute over the copyright of a suspect image occurs, the image is first processed by watermarking. If the watermark can be retrieved from the suspect image, the copyright can then be confirmed; otherwise, the watermark then serves as the fingerprint and is processed by fingerprinting. If a match in the fingerprint database is found, then the suspect image will be considered a duplicated one. Because the proposed scheme utilizes both watermarking and fingerprinting, it is more robust than those that only adopt watermarking, and it can also obtain the preliminary result more quickly than those that only utilize fingerprinting. The experimental results show that when the watermarked image suffers slight attacks, watermarking alone is enough to identify the copyright. The results also show that when the watermarked image suffers heavy attacks that render watermarking incompetent, fingerprinting can successfully identify the copyright, hence demonstrating the effectiveness of the proposed scheme.
1. Introduction
Many researchers [1–17] have been engaged in finding the solution to protecting copyrights of digital images, which may be duplicated and distributed over the Internet without the authors’ permission. Generally speaking, there are two approaches to discovering image copyright infringement. One is watermarking [1–11] and the other is fingerprinting [12–17]. The main idea of watermarking is to embed a piece of information (i.e., watermark) in the host image. If a similar watermark can be retrieved from a suspect image, it is then considered a duplicated one. On the other hand, the principle of fingerprinting is to extract unique features (i.e., fingerprints) from both the host image and the suspect one for comparison. If their fingerprints are similar, the ownership of the image can then be confirmed.
There are some general considerations on the two techniques, including the processing time and robustness. In terms of processing time, watermarking is more efficient because fingerprinting needs extra time to compare the image’s fingerprint with those stored in the database. If the database is large, it will be very time consuming. On the other hand, fingerprinting is generally more robust [18, 19] because when a watermarked image suffers some image processing operations that modify the content of the image, the embedded watermark will usually be damaged or even destroyed. On the contrary, since normal image processing does not destroy the features of an image, the fingerprint of the image can therefore be preserved. In summary, fingerprinting is more robust whereas watermarking is more efficient. If the complementary natures of two approaches can be utilized properly, a robust and efficient scheme can then be developed to identify copyrights.
This paper proposes a novel scheme that combines the two techniques to identify copyrights for color images. The proposed scheme generates from the image a fingerprint, which also serves as the watermark. The watermark is then embedded in the host image to produce a watermarked image. When there is a dispute over the copyright of a suspect image, the suspect image will first be processed by watermarking, which tries to retrieve the watermark from the suspect image. If the watermark is identified, the copyright is confirmed at this stage; otherwise, the image will then be processed by fingerprinting, which utilizes the retrieved watermark as the fingerprint and compares it with those stored in the database. If a match is found, then the suspect image will be considered a duplicated one.
2. Related Background
The proposed scheme utilizes a special technique called image secret sharing (ISS), whose details can be found in the paper [20] we published in 2008. The following briefly describes the main idea of the technique utilized by the proposed scheme.
The ISS generates a share image from two images. In the proposed scheme, the two images are the logo image and the feature image (as depicted in Figure 1). The logo image can be any identifiable image. The feature image is generated from the input image as follows. First, the input image is split into nonoverlapping 8 × 8 blocks. Then, the 2D DWT is applied to each block to generate four subbands, LL2, LH2, HL2, and HH2. An example of 2D DWT is shown in Figure 2. Finally, the coefficients in the LL2 subband of each DWT block are used to generate the feature image. The ISS then generates a share image from the logo image and feature image. The share image will be used as the fingerprint of the input image by the scheme. The share image also serves as the watermark to be embedded in the host image. The benefit of the ISS scheme is that performing the XOR operation on the feature image and the share image will restore the logo image, which can then be used to identify the copyright.
3. The Proposed Copyright Identification Scheme
The proposed scheme contains two phases: the fingerprint and watermarked image generation phase and the authentication logo detection phase. The former phase extracts features from the host image, which, along with a logo image, is used to generate the fingerprint. The fingerprint also serves as the watermark, and the phase embeds it in the host image to produce a watermarked image. On the other hand, the latter phase extracts features and retrieves the watermark from the suspect image. The extracted features and the retrieved watermark are utilized to restore the logo image, which is used to identify the copyright. If it fails, the retrieved watermark then serves as the fingerprint and is compared with those in the database to determine if the suspect image is a duplicated one.
The fingerprint and watermarked image generation phase (shown in Figure 3) works as follows. In the beginning, feature extraction extracts the features of the host image and then logo scrambling disarranges the authentication logo to a scrambled logo image. After that, fingerprint generation takes as input the extracted features and the scrambled logo to generate the fingerprint. Finally, the fingerprint serves as a watermark and is embedded in the host image, which becomes a watermarked image. The fingerprint is also stored in a database for later use in the next phase.
The authentication logo detection phase (shown in Figure 3) checks the watermark first and, if necessary, the fingerprint next. In the beginning, watermark retrieval regains the watermark from the suspect image. Next, the features of the suspect image are extracted by feature extraction. After that, logo restoration takes as input the retrieved watermark (the expected fingerprint of the suspect image) and the extracted features to recover and rearrange the scrambled logo to restore the authentication logo. The phase ends if the accuracy rate of the restored logo determined by logo comparison is high enough; otherwise, the process proceeds to retrieve the next available fingerprint from the database and then returns to logo restoration, which takes as input the retrieved fingerprint instead of the extracted watermark. The phase restores the logo from the retrieved fingerprint as well as the extracted features and proceeds to logo comparison. The looping process continues until the authentication logo is discovered or no fingerprint is available.
3.1. Fingerprint and Watermarked Image Generation Phase
The following paragraphs detail the stages in the fingerprint and watermarked image generation phase, including feature extraction, logo scrambling, fingerprint generation, and watermark embedding.
3.1.1. Feature Extraction
The feature extraction stage takes a color image as input and then extracts its features. The stage has two substages, sampling and feature generation. During sampling, the stage first transforms the RGB image to the YCbCr color space [21, 22]. Then, it partitions each of the three channels into several nonoverlapping blocks of size 8 × 8 (hence, a color image of size will result in nonoverlapping blocks). After partition, for each row of the corresponding blocks in the three channels, the stage takes the first four samples from the Y channel, the next two from the Cb, and the last two from the Cr (as depicted in Figure 4) to generate new packed blocks, each of size 8 × 8.
After sampling, the stage enters its second substage, feature generation. For each of the packed blocks, the stage applies 2D DWT to the block, resulting in four coefficients in the LL2 subband. Next, the stage computes the average (denoted by ) of the four coefficients and then obtains a feature type according to the relationship of the four coefficients and the average as expressed in (1). Consider
According to , , and the mapping table shown in Table 1, a feature share (called FT-share) of size 2 × 2 is determined for each block. The FT-shares represent the features of the input color image. They are assembled to form the feature image.
The steps of the feature extraction stage are listed in Algorithm 1.
| ||||||||||||||||||||||||||||||
3.1.2. Logo Scrambling
In order to disperse the intensity of attacks, the proposed scheme adopts Torus automorphism [23] to scramble the authentication logo. The stage uses a predetermined key, , and the following equation to scramble the logo. Consider where (, ) are the coordinates in state t and is the coordinate size of the given image. Figure 5 shows an example of the authentication logo scrambled four times with , .
3.1.3. Fingerprint Generation
The proposed scheme uses ISS (mentioned in Section 2) to generate the fingerprint. It determines a FP-share for each FT-share (generated in feature extraction) according to the color of the corresponding pixel in the scrambled logo by looking Table 1 up. For example, if the feature type is 2 and the FT-share is the same as the one in Table 2, then the FP-share will be either one of the two FP-shares in Table 2 according to the color of the corresponding pixel in the scrambled logo. After every FP-share is determined, the stage gathers all the FP-shares to form the fingerprint image. The fingerprint also serves as the watermark in the next stage.
3.1.4. Watermark Embedding
The watermark embedding stage uses the resulting fingerprint image as a watermark and then embeds it in the host image. Generally speaking, for the RGB color space, the human visual system is more sensitive to the G channel than to the other two [22–28]. Therefore, the proposed scheme embeds the watermark into the less sensitive R and B channels of the host image. To be more specific, there are four areas: two in the R channel ( and ) and two in the B channel ( and ), used for watermark embedding (see Figure 6). The stage applies the 2D DWT to the R and B channels of the host image and next applies the 1D DWT to the resulting HL2 and LH2 to obtain the four blocks, , , , and .
The proposed scheme embeds the watermark into the image by adjusting the coefficients in the to blocks according to a predefined . The value of affects the robustness and the quality. The larger the is, the more robust the embedded watermark is, but the worse the quality of the watermarked image is. Figure 7 illustrates the adjustment of the coefficients. If the watermark bit is 1 and the current coefficient is between and , then the coefficient will be adjusted to be or , whichever is closer to the current coefficient. On the other hand, the adjusted coefficient will be or if the watermark bit is 0.
The embedding process is described as follows. First, the stage uses the current coefficient and the predefined to calculate S, sign, and according to (3), (4), and (5), respectively. In the equations, sign indicates that the is positive or negative; is the remainder for watermark bit value 0 and, similarly, for watermark bit value 1. Then, the stage determines the C_Low and C_High according to the value of the watermark bit. If the value is 0, (6) will be used, otherwise, (7). Finally, the stage adjusts the coefficient to according to (8). Consider
The steps of the watermark embedding stage are listed in Algorithm 2.
| ||||||||||||||||||
3.2. Authentication Logo Detection Phase
The phase is activated when a dispute over the copyright of a suspect image occurs. The following details the stages in the phase, including watermark retrieval and logo restoration.
3.2.1. Watermark Retrieval
The stage regains the watermark from the suspect image. First, the stage applies the DWT to the R and B planes of the suspect image in the same way as that in watermark embedding to obtain the embedding blocks, to (refer to Figure 6). Then, for each coefficient of the –blocks, it obtains the watermark bit according to where is the same as that in (3).
Finally, the watermark (which is supposed to be the fingerprint of the image) is restored by assembling every for each coefficient in the – blocks.
Algorithm 3 lists the steps of the watermark retrieval stage.
| ||||||||||||||
3.2.2. Logo Restoration
The stage restores the authentication logo. As shown in Figure 8, it has four substages: scrambled logo recovery, logo unscrambling, logo enhancement, and logo resizing.
Scrambled Logo Recovery. After retrieving the fingerprint image (i.e., watermark) by watermark retrieval and extracting the feature image by feature extraction, the substage performs XOR operation on each pixel of the fingerprint image and the corresponding pixel of the feature image to retrieve the scrambled logo. Because both of the two images are black-and-white, each pixel is either 0 (black) or 1 (white). Therefore, the substage simply performs bitwise XOR operation on the two images and obtains a scrambled logo.
Logo Unscrambling. As mentioned in Section 3.1.2, the logo was scrambled before it is used to generate fingerprint in the fingerprint and watermarked image generation phase. The substage adopts (10), which is the inverse equation of (2), to rearrange the scrambled logo and restore the logo. Consider
Logo Enhancement. The substage enhances the restored logo by erosion and dilation. Erosion removes pixels on object boundaries in an image and therefore can be used to remove smaller islands in the image; dilation, on the other hand, adds pixels to the boundaries of objects in an image and hence can be used to remove bright areas from the image. The substage performs erosion followed by dilation once, which is illustrated in Figure 9.
Erosion is performed by resetting each of the pixels according to (11). That is, if one of the neighbors is black, then the current pixel will be set to black; otherwise, it will be set to white. Let L(x, y) be the current pixel value to be determined and , , and its neighbors; then
Dilation, on the contrary, resets the neighbors of each pixel rather than the pixel itself. If L(x, y) is black; the substage sets all of its neighbors, , , and , to black; otherwise, the neighbors remain unchanged.
Logo Resizing. The proposed scheme adopts ISS to retrieve the authentication logo, which causes pixel expansion because one logo pixel is mapped to a share of four pixels (mentioned in Section 3.1.3). As a result, the retrieved logo from the previous stage will be larger than the original one. To resize the logo to its original size, the substage partitions the enhanced logo into several blocks of size 2 × 2, each of which is then reduced into one pixel with value according to the following rule:
4. Experimental Results
Two kinds of experiments were conducted to prove the effectiveness of the proposed scheme. The first experiment shows the robustness of our scheme and the other demonstrates the capability of unique identification. In the experiments, the authentication logo used to generate the watermark (fingerprint) is shown in Figure 10.
Two common measurements used to estimate the robustness of our scheme are described as below.
(1) Peak Signal to Noise Ratio (PSNR). The measurement to estimate the color image quality after image processing is a variant version of normal PSNR [29]. The variant PSNR listed below does not consider the influence of the green channel because the channel is not modified by our scheme. Consider where MSE is the mean square error between the original image and the modified image, which is defined as follows: where represents the original pixel value and denotes the modified pixel value.
According to the definition of PSNR, the higher the value is, the better the quality of the modified image is. Generally, if the PSNR is greater than 30 dB, the quality of the modified image is acceptable.
(2) Accuracy Rate (AR). The measurement shown below is used to evaluate the correctness of the logo after it has been restored. Consider where NP is the number of pixels in the original logo and CP is the number of correct pixels obtained by comparing the pixels of the original logo with the corresponding ones of the restored logo. Figure 11 shows the restored logos with different AR values. As can been seen, restored logos with AR higher than 81% (the ones in the upper row) still can be visually recognized whereas the restored logo with AR equal to 76% is hard to identify. However, according to the description in Section 4.2, if AR is higher than 75%, the scheme still can identify the copyrights of the image.
4.1. Robustness Experiments
The experiments proved our scheme is robust to different kinds of attacks. The test images used in the experiment, including “Lena,” “Mandrill,” “Sailboat,” and “Peppers,” are shown in Figure 12. The commercial image processing software “Adobe Image Photoshop CS” was used to simulate several kinds of image attacks, some of which for “Lena” are shown in Figure 13.
(a)
(b)
(c)
(d)
The experimental results of the test images are shown in Table 3. As mentioned above, if AR is less than 75%, the scheme cannot identify the copyrights of the image. Table 3 shows that our scheme failed to verify the copyrights for the images suffering the heavier attacks of JPEG, contrast, Gauss blurring, and scaling in watermark verification. Nevertheless, the duplications of those images can all be determined in fingerprint verification. In summary, our scheme can identify the copyrights of the suspect images under moderate attacks in watermark verification and determine the duplications of those suffering the heavy attacks in fingerprint verification.
Moreover, there were 100 attacks in total and 53 of them resulted in an AR value higher than 75% in watermarking verification. That is to say, the copyrights of the 53% of the attacked images can be successfully identified in watermark verification, and hence only 47% of them need fingerprint verification.
4.2. Uniqueness Experiments
The experiment showed that our scheme has the capability of unique identification and is able to distinguish a copyrighted image from different ones. The four copyrighted images (with embedded watermarks) in Figure 12 along with seven unwatermarked images (Figures 14(a)–14(g)) were processed by our scheme to identify the copyright. The stored fingerprints of the watermarked images were used in fingerprint verification to restore the logos for all of the images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
The results shown in Table 4 demonstrated the extraordinary unique identification capability of our scheme. It can be clearly seen that all the restored logos except the ones of the copyrighted images (those on the rightmost side) are unrecognizable, which proves that our scheme is actually able to distinguish a copyrighted image from different ones. It will not mistakenly identify the copyrights of an unwatermarked image.
Moreover, the resulting AR values of all the unwatermarked images are all lower than 75%. Hence, it is reasonable for our scheme to confirm the copyright when AR is higher than 75%.
4.3. Discussion and Comparison
The fingerprint extracted from an image is more robust than the watermark embedded in the image. This can be seen from Figure 15, which shows the average AR values of the images suffering different attacks for watermarking and fingerprinting. The AR values for fingerprinting are all higher than those for watermarking. When the image undergoes image processing operations that heavily damage the embedded watermark, the extracted fingerprint still can survive the attacks. However, because linear comparison is a computationally intensive process, fingerprinting is costly in time if there are many fingerprints in the database.
(a)
(b)
(c)
(d)
(e)
(f)
Table 5 shows the processing time for watermark extraction and logo restoration and comparison in our scheme. The experiments were carried out on a computer equipped with the following hardware and software: CPU: 3.16 GHz Intel(R) Xeon(R) CPU E3120, RAM: 4 GB, OS: Windows 7, Computing language and environment: MATLAB.
As Table 5 shows, it takes about 0.21 seconds to extract the watermark from the input image and 0.029 seconds to restore and compare the logo. Therefore, the total processing time (in seconds) of our scheme can be expressed by where represents the number of the fingerprints retrieved from the database. If the copyright can be identified in watermark verification, is 0. That is, our scheme only needs 0.24 seconds in the best case. Otherwise, our scheme needs additional 0.029 seconds for each retrieved fingerprint in the database.
Because our scheme combines watermarking and fingerprinting techniques, it can be as efficient as pure watermarking schemes and also as robust as pure watermarking schemes. A pure watermarking scheme is very efficient because it only needs to make one watermark comparison to verify the copyright. However, it may not be as robust as a pure fingerprinting scheme when dealing with images that have suffered heavy attacks. Table 6 shows the comparison of our combined scheme and the other two pure schemes. If the copyright of the input image can be identified by watermarking verification, our scheme can be as efficient as pure watermarking schemes. If watermarking verification fails to identify the copyright, our scheme is still able to determine duplication in fingerprinting verification, which makes our scheme as robust as pure fingerprinting schemes.
5. Conclusion
This paper presented a copyright identification scheme that takes advantage of the complementary nature of digital watermarking and fingerprinting. The experimental results showed that when the watermarked image suffers moderate attacks, watermarking verification alone is enough to identify the copyright, and there is no need for fingerprinting verification. In other words, the proposed scheme can identify the copyright efficiently in this situation. On the other hand, the experimental results also showed that when the watermarked image suffers heavy attacks that render watermarking verification incompetent, fingerprinting verification, although more time consuming, can successfully determine the duplication, hence demonstrating the robustness of the proposed scheme.
One distinguishing characteristic of the proposed scheme is that it does not need a separate watermark for watermarking verification and a separate fingerprint for fingerprinting verification. The proposed scheme extracts features from the input image to generate the fingerprint, which also serves as the watermark. Hence, only one piece of information is needed for both watermarking and fingerprinting verifications.
To further improve the scheme, retrieving the stored fingerprint image from the database to restore the correct authentication logo more quickly is worth studying. When there are more than one fingerprint image in the database, the original fingerprint of the host image must be correctly retrieved; otherwise, the correct authentication logo cannot be restored. Retrieving every stored fingerprint image to restore an authentication logo for comparison is very time consuming. The scheme should provide a more efficient way that is able to find the proper one in less time, which is the future work of the research.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
Financial support of this study by Tatung University, Taipei, Taiwan, under Grant B100-I07-036 is gratefully acknowledged.