Abstract

An efficient procedure for recovering spectral reflectance using an object’s tristimulus values under multi-illuminants is proposed by adapting with the characteristics of the testing sample to obtain the transformation matrix of pseudoinverse. Specifically, we propose the reference illuminants selection strategy and local sample weighted strategy to obtain the optimal transformation matrix under multi-illuminants condition. Selecting the reference illuminants are based on the result of the spectral angle mapper (SAM) statistics. The number of the selected local training samples and the weighted local samples can be determined by using the multicolor space Euclidean distance. To compare the experimental results, the proposed method significantly increases the spectral and colorimetric accuracy for the spectral reflectance recovery process.

1. Introduction

The object’s spectral reflectance is almost the definition of “fingerprinting” that predicts accurately the object appearance under arbitrary illuminants and observers. The most accurate and effective information to represent an object color by its spectral reflectance is highly desirable, since it is essential in common application scenarios, for example, printing inspection, disease diagnosis, textile color matching, and computer vision [14]. Normally, the spectral reflectance could be directed to acquire from the spectrophotometers and spectral cameras. Unfortunately, portability, complexity, and expensiveness of these devices limit their own availability. The most used colorimetric values, instead, can be easily procurable by digital still cameras, smartphone, and colorimeters.

The colorimetric values such as RGB or CIE XYZ tristimulus values, obtaining from three channels, only record color information under fixed viewing conditions. There are some cases in which this representation is not enough, because it is heavily dependent on the illuminant. So, the colorimetric values still vary significantly with the illuminants. Although the computation of colorimetric values is uniquely made from the object’s spectral reflectance, the problem in the spectral reflectance recovery calculating from the colorimetric values is usually ill-posed. Many different mathematical methods are still widely studied to recover the spectrum, for example, the pseudoinverse method (PI) [5], principal component analysis (PCA) [6], matrix R method [7], non-negative matrix transformation (NNMF) [8], simulated annealing [9], compressive sensing [10], simplex method [9], and so on. Of all these methods, the PI method is an uncomplicated and straightforward solution for the ill-posed inverse problem, which shows more clearly the connection between colorimetric values and the corresponding spectrum.

In fact, spectral recovery from a set of the corresponding colorimetric values under a specified illuminant-observer condition limits only three available dimensions, which leads to a noticeable insufficiency for spectral reflectance recovery. The colorimetric values under multi-illuminants have inspired researchers to start using several different sets of the corresponding values for the spectral recovery process. Schettini and Zuffi [11] exploited genetic algorithms to spectrum recovery from the CIE XYZ tristimulus values under one or multi-illuminants on the premise that basis functions and cardinalities were investigated. Abed et al. [12] proposed the different color spaces of lookup tables (LUTs) using a novel interpolation strategy for spectrum recovery under illuminate D65 and illuminate A; they stated that the more color coordinates reconstructed LUTs, the better the spectrum recovery results could be obtained. Harifi et al. [13] initially applied the nonlinear regression method to virtually increase the number of tristmulus values and then used six eigenvectors in the spectral recovery process. Amiri and Amirshahi [8] increased virtually the CIE XYZ tristimulus values under another illuminant and then recovered the spectral reflectance using of six eigenvectors by adopting the PCA or NNMF method, respectively. Zhang et al. [14] initially predicted CIE XYZ tristimulus values from camera responses values according to different illuminants and then recovered the spectral reflectance through the PI method. However, the aforementioned methods are available for spectral reflectance recovery from tristimulus values under predefined reference illuminants (such as illuminant A and illuminant D65) and treat equally each of the training samples to effect on the spectral recovery process under multireference illuminants.

This study proposes a more accurate recovering spectral reflectance method from tristimulus values under multi-illuminants that solves the problems mentioned above.

The novelty of our proposed method is concerned with the reference illuminant selection strategy and local sample weighted strategy under the multi-illuminants condition. The experimental results of the proposed and traditional methods are compared to evaluate the spectral and colorimetric accuracy between the actual and estimated spectra.

2. Mathematic Background and Method

The CIE XYZ tristimulus values under multi-illuminants are simply computed by the following equations:withwhere denotes the object’s reflectance spectrum; represents the factor of standardization; , , and denote the color matching functions of the CIE standard colorimetric observer; denotes the reference illuminants of index ; denotes the object color tristimulus values under a given reference illuminant ; and the superscript denotes the matrix transpose. Equation (1) can then be represented in matrix notation as follows:where is the combining matrix of tristimulus values under multi-illuminants and is the coefficient matrix involving the reference illuminants and the CIE color matching functions. For arbitrary color of the combining matrix from tristimulus values under multi-illuminants, the recovered spectra can be calculated to obtain from the inverse matrix directly. So, this spectral recovery process immediately implements transformation from tristimulus value space to spectral reflectance space called the direct spectral recovery method [15]. Since is an underdetermined matrix, the recovered spectra are obtained from the pseudoinverse matrix , which inevitably causes large calculation error on ill-posed inverse problems. The recovery of spectral reflectance using the PI method is simply calculated by the following equation:where the superscript “+” is the matrix pseudoinverse.

Equation (4) creates a linear conversion between tristimulus values under multi-illuminants and the spectral reflectance, so it is assumed to be a straightforward and resultful solution. When the transformation matrix is calculated, the recovery process directly yields a single linear relation. It must be accepted that for the spectral recovery process by the implementation of the PI method, each of the training samples has made an equal difference in the formation of , so it leads to optimization for the samples fully, but not for each individual sample. Hence, the standard pseudoinverse method leads to large calculation error between the actual and estimated spectra and generates an imprecise result. Obviously, the more similar the spectral reflectance between the testing samples and the training samples is, the more accurate the results can be generated due to the more linearly relation [16, 17]. Theoretically speaking, if the matrix can be formed by the testing sample’s own characteristics, the performance of the recovered spectra would be calculated optimally. In this work, to obtain the matrix adapting with the characteristics of the testing sample, the local sample weighted strategy is proposed for calculating the spectral reflectance recovery. The local sample weighted strategy involves selection of the optimal training samples and weighting of the local selected samples. The local optimal training samples based on the testing sample's own characteristics are selected, which use the multicolor space Euclidean distance between the testing sample and the training samples. Specifically, the multicolor space Euclidean distance between each testing sample and the training samples can be calculated as follows:where , , and denote the tristimulus values of the testing sample under the th reference illuminant; demonstrates the number of the training samples; , , and denote the th training sample’s tristimulus values under the th reference illuminant; and refers to the multicolor space Euclidean distance between the testing sample and the th training samples under the multi-illuminants condition. After that the training samples arrange in the increasing order based on the values. The local optimal training samples have been obtain to select the neighboring samples from the whole training samples. Meanwhile, different local training samples should choose different weighting efficients since the larger the weighting efficient a certain training sample chooses, the stronger the influence the matrix determinates. So, the weight coefficient, depending on the inverse multicolor space Euclidean distance, can be calculated aswhere the subscript denotes the th local optimal training samples; denotes the multicolor space Euclidean distance between the testing sample and the th sample of the local optimal training samples; and is used in this study. Clearly, more neighboring training samples to the testing sample would generate the larger value. This mathematical symbol wk is shown in the galley proof. The weighting matrix is a diagonal matrix defined as

Indeed, the designing method has selectively controlled the influence on the formation of the matrix based on the characteristics of the testing sample, which varies from one testing sample to the other one. Each of the testing samples should have its own the transformation matrix rather than a unique transformation matrix for the whole testing samples, which generates a more precise estimation result.

3. Experiment and Procedure

In this study, three different datasets were selected that include the Munsell Chips [18], ColorChecker SG, and Vrhel dataset [19]. The Munsell Chips consist 1269 spectral color chips in Munsell Matte Color Book. The 140 chips of the ColorChecher SG use a X-rite i1 pro spectrophotometer to measure personally. The Vrhel dataset contains 354 samples of reflectance spectra. The spectral reflectance function is proved to sample at 10 nm intervals without impacting greatly the mathematical precision [20]. So, the spectral reflectance of three different dataset samples range from 400 nm to 700 nm at 10 nm intervals. The Munsell Chips were selected as the training samples in this study, and their transformation matrix was used to recover three different testing samples.

To illuminate the advantages of the proposed method, the standard pseudoinverse method (method PI) and principal component analysis embedded weight regression technique suggested by Amiri and Amirshahi [8] were compared with the results of the spectral recovery from tristimulus values under multireference illuminants. The root mean square error (RMSE), goodness of fit coefficient (GFC), and CIELAB color difference (∆Eab) were selected as the evaluation of the spectral recovery accuracy [21], while all the methods of the spectral reflectance recovery perform experimental simulation by the Matlab software.

The dominant spectral power distributions (SPDs) of the selected reference illuminants and light sources were of various distributions and were relatively smooth, which did not have considerable spiky radiance. So, the CIE illuminants (A, B, C, D50, D55, and D65) and the two actual light-emitting diode (LED) light sources (LED1: Sylvania Concord 2048794 and LED2: Photon Star CS5) [22] were selected as the reference illuminants so as to evaluate spectral recovery accuracy in this study. All the CIE illuminants and LEDs were sampled at a range from 400 nm to 700 nm at 10 nm intervals. To evaluate the performance of the proposed method, it is critical to determine the type of the reference illuminants, that is, what is the optimal type of reference illuminants to improve the spectral reflectance recovery accuracy. As Zhang et al. [14] discussed previously, a trend can be observed: the more similar the spectral power distributions (SPDs) of two selected reference illuminants are, the lower the spectral recovery accuracy is. However, this point lacks of a clear numerical explanation to verify. This study adopted the spectral angle mapper (SAM) algorithm [23, 24] to calculate the similarity of the reference illuminants as the reference illuminants selection strategy, shown aswhere and denote the SPDs value vectors of the reference illuminant of index and the reference illuminant of index , respectively. This method determines the SPDs similarity to treat the two reference illuminants as vectors with treating equally to each dimension and calculate the angle between the two reference illuminants, which do not adopt the vector length but rather the vector direction [15].

4. Results and Discussion

The proposed spectral reflectance recovery method mainly involves two optimal parameters, namely, optimal parameter of the reference illuminants and optimal parameter of the local sample weighted method. This study first investigated two optimal parameters of the spectral recovery process and finally evaluated the spectral reflectance recovery accuracy of the proposed compared with the traditional methods.

4.1. Optimal Parameter of the Reference Illuminants

The spectral reflectance recovery accuracy can be affected by the number and type of selected reference illuminants. Theoretically, increasing the number of reference illuminants can increase the spectral recovery accuracy, but obtaining extra tristimulus values by the colorimetric device would be not easily procurable. Meanwhile, according to the conclusion made by Schettini and Zuffi [11], the result of the spectral recovery under the three reference illuminants is less to improve the computing accuracy than under the two illuminants. In this study, the number of reference illuminants determined to two illuminants. The type of reference illuminants selected optimally are the relatively smooth illuminants rather than the considerable spiky illuminants; this is because spectral recovery accuracy was low when one or two spiky illuminants were adopted as reference illuminants [14]. First, all the samples of the Munsell Chips, ColorChecker SG, and Vrhel dataset were calculated numerically to obtain the CIE 1964 XYZ tristimulus values under the CIE illuminants (A, B, C, D50, D55, and D65) and the two actual light-emitting diode (LED) light sources (LED1: Sylvania Concord 2048794, LED2: Photon Star CS5), which illustrated to select optimally the type of reference illuminants. In this study, we considered the Munsell Chips as the training samples to estimate the spectral reflectance of the Munsell Chips, ColorChecker SG, and Vrhel dataset under two different illuminants with the corresponding CIE 1964 XYZ tristimulus values. Table 1 shows the mean root-mean-square error (RMSE) between the actual and reconstructed spectra for the Munsell Chips, ColorChecker SG, and Vrhel dataset under two illuminant combinations. First, it is noted that the CIE 1964 XYZ tristimulus values under two different illuminants have a very significant influence on the spectrally recovered result. Second, Table 1 illustrates the spectral reflectance accuracy for the Munsell Chips presented better than for the ColorChecker SG and Vrhel dataset. As Babaei et al. [5] proposed in a previous study, the best optimum condition would be obtained when the testing sample is a number of the training samples. Third, the similarity of the two reference illuminants also affects spectral estimation accuracy. Table 2 shows the spectral angle mapper (SAM) statistics for two reference illuminants. As the spectral curve shape of the two reference illuminants is more dissimilar between each other, the spectral recovery accuracy will be better, as shown in Tables 1 and 2. Combining all the experimental results of Tables 1 and 2, this study selects finally the two illuminants D65 and A as the reference illuminants.

4.2. Optimal Parameter of the Local Sample Weighted Method

The number of the selected local training samples and the weighted local samples can also influence the accuracy of the spectral reflectance recovery. To analyse these issues, the spectral reflectance of the Munsell Chips, ColorChecker SG, and Vrhel dataset was recovered by using the multicolor space Euclidean distance and the weighting matrix under the reference illuminants A and D65 with different numbers of the local Munsell Chips as the training samples. The mean RMSE between the actual and recovered spectra under different numbers of the local Munsell Chips was computed. It is noted that in selecting, dynamically, the suitable local samples, we always adopt the strategy that if the testing sample itself is included in the Munsell Chips, then this reflectance is removed from the training samples. The relationship between the mean RMSE of the Munsell Chips, ColorChecker SG, and Vrhel dataset and the number of local training samples is shown in Figure 1. It is found that the mean RMSE initially decreases with the increase in the number of the local training samples and trends basically stable in the end. To achieve the better spectral reflectance accuracy, sufficient training samples can be selected. So, we selected adaptively 100 local training samples from the Munsell Chips for the spectral recovery process.

4.3. Accuracy Evaluation of the Proposed Method

To evaluate the colorimetric and spectral performance of the proposed method, the proposed method was implemented to compare with the pseudoinverse method (method PI) and the Morteza Maali Amiri’s method (method WRPCA) [8]. The statistic comparison results among PI, WRPCA, and the proposed method are summarized in Table 3. First, it is easy find that the proposed method obviously outperforms the PI and WRPCA for the three testing datasets. Second, results in Table 3 suggest that all the methods for Munsell Chips implemented better than for the ColorChecker SG and Vrhel dataset. This finding shows that the training sample has the better performance to recover itself than other testing samples. Second, Table 3 indicates that the proposed method presents the spectral reflectance with the highest accuracy compared with the other methods.

To further accurately evaluate the performance of the proposed method, the average spectral residuals between recovered and measured spectra for the three datasets are shown in Figures 24. In each figure, PI, WRPCA, and the proposed methods are contrasted for the three different datasets: the Munsell Chips (Figure 2), the ColorChecker SG (Figure 3), and the Vrhel dataset (Figure 4). By the analysis of Figures 24, it is easy to note that the average spectral residuals are more accurate to the middle of the wavelengths than both ends. The phenomenon is consistent with the conclusion made by Wu et al. and is attributed mainly to the human visual system that obtains the XYZ tristimulus values for three methods being calculated by the CIE 1964 standard observer [15].

In order to illustrate the performance of the proposed method, the recovered results of the spectral reflectance for two randomly selected samples from the Munsell Chips are shown in Figures 5(a) and 5(b), ColorChecker SG are shown in Figures 5(c) and 5(d), and Vrhel dataset are shown in Figures 5(e) and 5(f). Compared with the PI and WRPCA methods, the performance of the proposed method is more accurately illustrated in Figure 5.

5. Conclusions

A method for the spectral reflectance recovery from tristimulus values under multi-illuminants was presented by adapting with the characteristics of the testing sample to obtain the transformation matrix of pseudoinverse. To obtain the optimal transformation matrix, the proposed method for spectral reflectance recovery mainly involves two optimal parameters: optimal parameter of the reference illuminants and optimal parameter of the local sample weighted method. Selecting the two illuminants A and D65 as the reference illuminants are based on the result of the spectral angle mapper (SAM) statistics for two reference illuminants. The number of the selected local training samples and the weighted local samples can be also determined for the spectral reflectance recovery.

The reflectance of the testing samples from the Munsell Chips, ColorChecker SG, and Vrhel dataset were used to evaluate the spectral reflectance accuracy for different methods in this study. The Munsell Chips were selected as the training samples. Meanwhile, the performance of three different methods, namely, PI, WRPCA, and the proposed method was assessed by the root-mean-square (RMSE), the goodness of fit coefficient (GFC), and CIE LAB color differences under illuminants F2 and F11. The spectral and colorimetrical recovery accuracy of the proposed method were compared with those of PI and WRPCA, and the results showed the proposed method is an effective spectral reflectance recovery method.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This study was supported by the Shandong Provincial Natural Science Foundation, China (ZR2017LF027, ZR2017PC020); Key Research and Development Program of Shandong Province (2018GGX106009, 2019GGX105016); Project of Shandong Province Higher Educational Science and Technology Program (J17KA178, J18KA332); Open Fund of National-Local Joint Engineering Laboratory for Digitalize Electrical Design Technology (NELDED2017K003); National Natural Science Foundation of China (NSFC) (11704162); Foundation of Xuzhou City (KC18002); and the Key Laboratory of Auxiliary Chemistry and Technology for Chemical Industry, Ministry of Education (KFKT2019-03).