Abstract
Influenced by the natural environment and preservation conditions, the frescoes in the monastery were damaged to some extent. In order to solve the fresco image restoration conversion and structural decomposition defects, an intelligent fresco digital image restoration technique based on machine learning algorithm is proposed. Firstly, the digital image information of mural is collected by a scanning program, and the data in the database is extracted. The mean filter template is used to restore the color of mural digital image, and the Gaussian template is used to restore the color of image details. The two are superimposed to ensure that the image is not affected by distortion and halo, so as to ensure the clarity of image and boundary. The depth learning model of local fuzzy feature restoration of mural digital image is obtained by using multimodal feature decomposition method, and the intelligent restoration of mural digital image is realized by this model. The experimental results show that this method can effectively repair the local fuzzy features of mural digital image. The repair precision is more than 95.7% and fills in the missing information from the image, and the image quality is good.
1. Introduction
As one of the more primitive painting forms in China, murals have a long history and culture [1]. The early murals in China were mainly painted in grottoes, halls, and tombs. They originated in the era of the Yellow Emperor and are usually used to record the lifestyle, habits, and inner wishes of the ancients. Some murals are used to describe early fairy tales [2]. Among them, the murals of the Saint Monk’s nunnery are rich and diverse, mainly including Buddha statues, Bodhisattva statues, major historical events, and custom paintings [3]. The murals of the Holy Monk’s nunnery mainly reflect the description of characters and events and the texture of folk decoration. They are the treasures of China’s national art [4]. With the passage of time, mural images are affected by natural factors, such as wind and sand erosion and natural decolorization, which will cause varying degrees of damage, and even lead to the gradual disappearance of the image [5]. With the rapid development of computer technology and information technology, at this stage, China has studied many image restoration technologies for mural image processing, such as wavelet coefficient image denoising technology and image sample block completion algorithm, which is of great significance for mural image restoration [6]. However, the construction cost of most mural image restoration technologies is high, and the effect in image conversion and structure decomposition is still poor [7].
In recent years, the rise of machine learning methods based on big data has brought new opportunities to this field: using deep learning methods, scholars can use the existing image database to train the computer, so that it can find features that cannot be captured by the human eye outside the shape and color and improve the accuracy of classification [8]. In fact, there are already deep learning network models such as AlexNet and GoogLeNet that can be competent for such work. In this study, the researchers applied these methods at the same time. In addition to the shape and color features, the researchers also used the Mogao cave mural database and applied the deep convolution neural network (DCNN) method to train the deep learning model Dunnet, which is specially used for the dating of Mogao cave murals. Its fault representation is now better than the traditional methods and other depth networks as a whole. Reference [9] proposed a depth neural network based on feature compensation to reconstruct superresolution images. This method extracts the high-frequency information of the original image and fuses it with the reconstructed superresolution image to improve the image quality. Reference [10] proposes a technique for reconstructing ancient murals by combining texture and structure. The degraded part is identified by creating a mask to complete the filling, and the structure and texture information of the mural is reconstructed by generating a sketch. Reference [11] studies the restoration of ancient painting images, which can be digitized and archived for future artists. The preprocessed image is applied to the restoration algorithm.
The above research has realized the restoration of fresco image, but for the restoration process of Sheng Sangan fresco image, the restoration effect is greatly influenced by noise. Therefore, this paper takes the fresco of Shengsenan as the object of restoration, based on the clear structure characteristics and restoration principle, and uses machine learning algorithm to achieve the restoration from image enhancement, color restoration, and deep learning model. Through this research, in order to contribute to the protection and inheritance of our country’s mural culture.
2. Analysis of Image Structure Characteristics of Murals in Saint Monk’s Nunnery
Huizhou Daoshi murals include religious murals and residential murals. At present, there are two famous murals: the mural of qilitou Saint Monk’s temple in Shexian county and the mural of Xiaoxi conglin temple. Among them, the mural of Saint Monk’s temple is the only mural of the Ming Dynasty in Anhui Province. Due to years of disrepair and no special management, the picture is seriously damaged, and the color fades. If we do not strengthen research and protection, important cultural heritage will be on the verge of extinction. The murals of Saint Monk’s temple and conglin temple have been recorded and commented in Shexian chronicle, sheshi leisure Tan, Shexian Range Rover, Huang Binhong’s anthology, Xin’an painting school, and other documents, but they have not been studied.
At the end of the Ming Dynasty, the philosophical thought and literary trend of thought showed a complex situation. The painting time of the Saint Monk’s nunnery mural was in the late Ming Dynasty. In addition to a certain degree of standardization and secularization, the contents of different religions and different schools were also integrated. The painting techniques of the Saint Monk’s nunnery mural were also very different from those of the previous religious murals, which concentrated on the painter’s thorough observation of life and rich imagination. It includes the following points: (1) it adopts the expression method of freehand brushwork, rather than the fine brushwork and heavy color commonly used in ancient murals. The modeling of characters pays attention to exaggeration and deformation. The structure is tight and loose, and the opening and closing are moderate and changeable. This modeling style is closer to the characteristics of Zhejiang style figure painting and should be the continuation of Zhejiang style. (2) It is outlined in ink without heavy colors. The modeling structure is accurate, the lines are vivid and changeable, the techniques are different, and the composition is diverse, which has the legacy of Wu Daozi. The characters have vivid posture, delicate expressions, loose, and real clothing lines. (3) There is no use of bone method. In the painting of green cypress, the shape of the trunk of green cypress is outlined with a little thick ink, and the middle is chapped and rubbed with light ink, which looks thick and vigorous. Qiu Jingao’s ancient cypress looks complete and full of charm, which not only reflects the author’s spiritual pursuit but also participates in Zen.
In order to achieve the best effect of the mural image restoration technology designed in this paper, this paper first analyzes the structure and characteristics of the mural image of the Saint Monk’s nunnery. The mural image of Saint Monk’s nunnery is shown in Figure 1.
As shown in Figure 1, the long-term climate impact caused hollowing and falling off of the mural image, resulting in mildew of the pigment layer of the mural and huge damage. In the early days, the murals of Saint Monk’s nunnery mainly existed on the dry wall soil. Due to the limitations of technical conditions and environment, the ancients usually painted a layer of grass mud on the wall and painted murals on the mud surface, resulting in the fragile overall image quality of the murals and a high probability of suffering from diseases.
Considering the limitation of mural storage environment and conditions, the possibility of mural image damage is greatly improved in the storage process. The phenomenon of cracks, falling off, and hollowing is common in the murals of Saint Monk’s nunnery, which is related to the pigments drawn by the murals and the composition of the murals themselves. According to the characteristics and damage causes of the mural images in the Holy Monk’s nunnery, the research and design of the mural image restoration technology can improve the effect of image restoration.
3. Mural Digital Image Restoration Based on Machine Learning Algorithm
3.1. Principles of Mural Digital Image Restoration
Import the mural image material of the target Holy Monk’s nunnery into the computer image processing system program, collect and transmit the information of each part of the image through the scanning program to the image information conversion program, and convert the image information into data representation according to relevant processing rules [12, 13]. Use the data classification standard to comprehensively and accurately summarize and sort them, save them in a specific image data folder, form backup data in the background, and save them in the system database to prevent the loss of system data caused by accidents [14]. The principle of mural digital image restoration is shown in Figure 2.
It can be seen from Figure 2 that the detection algorithm of machine learning method is used to detect and process the image data, clarify the relationship between the images after image cutting and decomposition and the corresponding processing order, and detect according to a certain arrangement law of image parameters [15, 16]. When the number of image materials is multiple, they can be judged in order according to the changes of image light, shadow, and angle. If it is a single image, it is arranged according to the corresponding position relationship of each part of the local image. Detect the image boundary; extract the image pixels, colors, and other parameters of the boundary part [17, 18]; compare with the parameter standard of the target processing level; and record the difference data in detail, so as to comprehensively carry out image equalization and color restoration. The image restoration process is shown in Figure 3.
According to Figure 3, the machine learning method is used to detect the image structure, detect the structural parameters of each image material, and vector simulate the target state of the image structure. Select appropriate structural parameters to describe specific image structural features, and obtain the confidence vector results of image parameter data through correlation matrix operation [19, 20], which is used for the transformation level detection of image-related materials or compressed domain. The research of this paper is to detect the transformation level of image compression domain. The confidence vector data of multiple image transformation levels are obtained by machine learning algorithm [21, 22], and then, the proportion of image compression parameters is obtained by secondary classification. After the parameter data of each part of the image is detected, analyzed, recorded, and sorted out, it is saved in the data manager of the image equalization and color restoration [23, 24] program of the image processing system to lay the data foundation for the process of mural digital image equalization and color restoration.
3.2. Image Reconstruction of Murals in Saint Monk’s Nunnery
The convolution neural network in the convolution neural network is a feedforward supervised learning network constructed by multiple convolution layers. It can be divided into input layer, hidden layer of output characteristic map, and output layer of output reconstructed image. After the initialization of convolution layer parameters in the network model, it is iteratively optimized and updated by error back propagation process. Using convolution neural network to reconstruct the mural image of Saint Monk’s nunnery can simplify the neural network while reducing the network training parameters, which has strong adaptability. The convolution neural network is used to reconstruct the mural image of the Holy Monk’s nunnery; that is, the convolution neural network is used to extract the prior knowledge from the image database and reconstruct the mural image of the Holy Monk’s nunnery according to the prior knowledge. The reconstruction process is as follows: (1)Process the image data of the murals of the Holy Monk’s nunnery and build an external image database. Collect the mural image of the Holy Monk’s nunnery, segment the initial image, use the degradation model to implement Gaussian filter fuzzy down sampling for the original mural image of the Holy Monk’s nunnery, and obtain the image low-resolution image block: where and represent the down sampling process and blur process, respectively; represents the image, low-resolution image, and initial image, respectively; and represents noise. The low-resolution image block of the degraded image is interpolated and enlarged to the size of the target image by bicubic interpolation method, which is used as the input of convolutional neural network.(2)The convolution neural network model is constructed. The image used for training is trained to obtain the mapping relationship between low-resolution image and high-resolution image. The structure is shown in Figure 4.
Based on Figure 4, the image extraction and feature representation of the mural in the Holy Monk’s nunnery: the feature of the low-resolution image block of the image is extracted by convolution neural network: where , , and , respectively, represent the weight of the convolution kernel of the first layer, the interpolated enlarged image, and the neuron bias vector. The convolution operation is carried out on and , and combined with , the image feature map of the mural of the Holy Monk’s nunnery is obtained. The characteristic graph is processed by ReLU activation function, and the maximum value in the result of 0 and convolution is the ReLU activation function.
After activation, the mural image of the Holy Monk’s nunnery is mapped from the low-resolution space to the high resolution space through nonlinear mapping. The expression is as follows:
where and represent the weight and offset of the second layer convolution kernel, respectively. (3)Image Reconstruction. Based on the input convolution neural network model, the digital reconstruction of the mural image block of the Holy Monk’s nunnery is completed.
Based on the feature map obtained in the above process, complete the image reconstruction of the mural in the Saint Monk’s nunnery: where represents the weight of the convolution kernel of the third layer.
The training process of convolutional neural network can be understood as parameter estimation and optimization. The solution obtained under the condition of minimum mean square error between the image reconstruction result and the initial image is the parameter optimal solution.
3.3. Design of Image Restoration Technology for Murals in Saint Monk’s Nunnery
Based on the above analysis of the principle of mural digital image restoration, combined with machine learning algorithm, this paper makes a specific research and design on the mural image restoration technology of Shengseng nunnery, as shown below.
3.3.1. Mural Digital Image Enhancement
When the color distortion of mural digital image will lead to the extremely poor visual effect of the image [25, 26], but the current RGB three channel image enhancement methods can not ensure that the enhancement effect of all regions in the image is the same, so the visual effect is still not guaranteed. In order to prevent the poor effect after enhancement, it is necessary to solve the pairwise relationship of three-dimensional vector in RGB space. In this space, represents not only the color of the image but also the brightness of the image. When the two vectors are proportional or inversely proportional, it indicates that the enhancement effect is good, and the ratio between them the brightness increased.
In order to ensure the same amplitude of image enhancement, it is necessary to enhance the illumination of all pixels in the image [27, 28]; that is, first extract the brightness component of the original image, that is, the component in the HSV color space, and then generate the brightness gain curve . The problem of extracting the brightness component is the maximum value processing of RGB three channels; that is, .
If the component is converted into a reflection component on the basis of Retinex, the expression is where represents the surround function of the reflection component and represents the brightness value of the image to be enhanced.
Gaussian filtering in the form of combining approximate mean filtering and fast mean filtering algorithm [29, 30] can arbitrarily and effectively improve the operation efficiency, and the brightness component of the image after operation has been qualitatively improved. In order to ensure that the enhancement process is not affected by safety noise, an enhancement adjustment factor needs to be added in the process of image enhancement, where is the amplitude of image enhancement adjustment by the adjustment factor. After adjusting the factor to enhance the brightness of the image, the component is .
The adjustment factor is extracted based on the sinusoidal characteristics. When the component is infinitely close to 0 or 255, the adjustment factor value is determined to be 0. At this time, the influence of noise on the image brightness can be ignored to ensure that the texture details of the image are obvious. Then, the brightness gain curve equation is
However, the pixel value of the enhanced image is infinitely close to 0. While enhancing the image, the region with high brightness is also enhanced. The region darkens due to excessive enhancement. Therefore, in the process of enhancement, the maximum brightness value needs to be extracted from the original image and the enhanced image, and the maximum value is used as the standard for enhancement. At this time, the formulas of , , and are, respectively: where represents the value of the image before enhancement, represents the value of the image before enhancement, represents the value of the image before enhancement, represents the value of the image after enhancement, represents the value of the image after enhancement, and represents the value of the image after enhancement.
After processing the image color in HSV space, the color in the image will not be distorted, and its color contrast and brightness are improved.
3.3.2. Color Restoration
Suppose that the Gaussian filter in the color restoration algorithm is , in which is the only parameter in the Gaussian filter. The size of parameter directly affects the color restoration ability of the detail area of the image. The smaller , the details of the dark area will also lead to distortion in other areas of the image at the same time of color restoration. The larger , the color restoration effect of the image is rising, and its compression ability is also declining and in the process of actually acquiring the image. Due to the halo phenomenon in the image with uneven illumination [31, 32], the data difference between the pixel value in the window and the central pixel is too large during the illumination component, resulting in the large difference between the filtered image and the original image. In order to eliminate the influence of halo on the image and improve the performance of color restoration algorithm, the Gaussian template is used to eliminate the halo. It can not only eliminate the halo very clearly but also ensure that the visual effect of the image is not affected. Adding parameter while using Gaussian template can also make the image color fidelity. Then, the expression of reflection component at this time is where represents the reflection part of the -th color component in the image, represents the original image of the -th color component in the image, and represents the Gaussian filtering window.
After obtaining the reflection component, add a color recovery factor to it and stretch it to the normal part to stop stretching.
To sum up, in order to better restore the color image, the mean filter template is used to restore the color, and the Gaussian template is used to eliminate the halo phenomenon for color restoration [33, 34]. The weighted values of the two color restoration parts are scored, and the results of , , and are as follows: where represents the value of the image after color restoration, represents the value of the image after color restoration, represents the value of the image after color restoration, represents the value of the image after color restoration, represents the value of the image after color restoration, represents the value that can be output after color restoration, represents the value of the image after color restoration and image enhancement, represents the output value of the image after the combination of color restoration processing and image enhancement processing, and represents the outputvalue of the image after the combination of color restoration processing and image enhancement processing.
The image only needs to meet the requirements of equation (9) to realize color restoration.
3.3.3. Mural Image Restoration
In combination with the above contents, collect the edge pixel set of mural digital image after color restoration and judge whether to build a deep learning model. If so, use the multimodal feature decomposition method to obtain the image sparse feature component [35, 36]. Combined with the superresolution fusion set, obtain the output result of local fuzzy restoration of mural digital image. If not, directly fuse the superresolution fusion set. The output result of local blur restoration of mural digital image is obtained. The flow chart of reconstructing mural digital image with pixel distribution matrix depth of field and repairing its local blur is shown in Figure 5.
Through the multimodal feature decomposition method, it is concluded that the depth learning model of local fuzzy feature restoration of mural digital image is where , , and , respectively, represent the block matching quantization set, sparse feature component, and template matching coefficient of mural digital image and represents the edge fuzzy pixel set. The multidimensional histogram structure model is constructed through the superresolution fusion set of mural digital images, and the correlation features match the detail information of different resolutions. It is concluded that the similarity feature quantity of mural digital image space is
Using the spatial visual feature distribution value of mural digital image, the , , and components of mural digital image local fuzzy feature restoration are calculated. The template matching values of mural digital image three-dimensional block frame point detection are , , and and , , and . The local fuzzy feature restoration results of mural digital image are output in the feature template according to Gaussian mixture model. So far, the mural digital image restoration based on machine learning is completed.
4. Experimental Analysis
In order to verify the effectiveness of the mural digital image intelligent restoration technology based on the machine learning algorithm proposed in this paper, the restoration results of this method, reference [9] and reference [10] are compared and analyzed. All experiments were completed on Windows 10 platform with Visual Studio 2017, and the computer was configured with Intel ® Core ™ m3-8100Y [email protected] GHz 1.6 GHz, and RAM 4 GB.
4.1. Experimental Preparation
Randomly select 10 digital images of murals in Holy Monk’s nunnery as the experimental object to study the application effect of this method in the digital image restoration of murals in Holy Monk’s nunnery. The spatial sampling resolution of the digital image of the murals in the Holy Monk’s nunnery is , the number of fuzzy information iterations is 200, the feature matching coefficient is 0.22, the correlation distribution set is 8, the initial pixel error matching set is 1, and the pixel spatial gain is 0.02.
4.2. Result Analysis
Randomly select a digital image of the mural of the Holy Monk’s nunnery as the original image, and repair it with three methods, respectively. The repair effect is shown in Figure 6.
In order to accurately analyze the image restoration results, the restoration results in Figure 6 are quantified; that is, the comparison results of image restoration precision of the three methods are obtained, as shown in Figure 7.
Through comprehensive analysis of Figures 6 and 7, it can be seen that although the methods of reference [9] and reference [10] have repaired the mural digital image to a certain extent, the restoration precision is not high. From the perspective of subjective visual effect, the image information of the mural digital image repaired by this method is clearer, which is significantly higher than that of reference [9] and reference [10], more than 95.7%. The highest restoration accuracy of the methods in reference [9] and reference [10] is 90.2% and 92.4%, which proves that the image quality and restoration effect of the mural digital image repaired by this method are better. This is because this method processes the mural digital image information through fuzzy information color restoration method, which improves the restoration accuracy.
Three groups of digital images of Holy Monk and conglin temple with different damage degrees are randomly selected, as shown in Figures 8–10 (the pictures are taken by the author himself).
Compare the repair execution time of the mural image shown in Figures 8–10 by the three methods, as shown in Table 1, respectively.
As shown in Table 1, in the restoration of three mural images of Saint Monk’s nunnery with different damage degrees selected in this experiment, the repair execution time of this method is short, and the maximum repair execution time is 13.2 s, which can complete the restoration of mural images in a short time, indicating that the visual effect of image restoration is more advantageous.
Randomly select three mural images of Saint Monk’s nunnery with different damage degrees. Firstly, preprocess the damaged area of the image, identify the damaged information and edge information of the image, eliminate the noise points of the mural image by filtering, extract the effective information of the mural image, set the gray value of the pixel of the image, suppress the nonlinear and smooth change of the image noise, and calculate the priority weight of the damaged area of the mural image in combination with the two-dimensional sliding template of the image. The formula is as follows: where represents the reliability value of mural image restoration, represents the value of mural image, and represents the pixel trust value of mural image restoration.
Through calculation, the priority weight of the damaged area of three damaged mural images is obtained. Next, regularize the blurred image of the mural, combined with the regularization parameters, set the iteration times of the repair technology, and obtain the convolution sum of the fuzzy kernel point spread function and the pixel blur of the mural image, so as to ensure the independent distribution of the pixels of the mural image. Through computer technology and information technology, calculate the PSNR value of three mural images. The greater the PSNR value, the better the quality of mural image restoration, the higher the definition of image quality, and the better the integrity of information saved after image restoration.
As one of the image quality evaluation standards, structure similarity measurement (SSIM) comprehensively measures the image restoration effect from the perspectives of brightness, contrast, and structure. The brightness is estimated by mean, the contrast is measured by standard deviation, and the structure similarity is measured by covariance.
The repair performance of the three methods is verified from two aspects of PSNR value and SSIM value. The results are shown in Table 2.
It can be seen from Table 2 that the maximum PSNR and SSIM values of the method in this paper are 27.79 and 98.69, while the maximum PSNR and SSIM values of the method in reference [9] are 25.27 and 98.56, and the maximum PSNR and SSIM values of the method in reference [10] are 26.89 and 98.58. It can be seen that the PSNR value of the mural digital image repaired by this method is higher than that of the methods in reference [9] and reference [10], which shows that this method can maintain more useful image information and reduce the repair error of local fuzzy features of mural digital image. The SSIM value of this method is higher than that of reference [9] and reference [10], which proves that this method can effectively restore the texture structure of mural digital image to a great extent and maintain the consistency of mural digital image structure.
5. Conclusion
To sum up, in order to improve the shortcomings of the traditional Saint monk’s nunnery mural image restoration technology, this paper makes a new research and design on the Saint Monk’s nunnery mural image restoration technology combined with machine learning algorithm. Based on the data in the mural digital image database, this paper carries out the overall and detailed color restoration of the mural image. By constructing the deep learning model of local fuzzy feature restoration of mural digital image, the intelligent restoration of mural digital image is realized. The experimental results show that the image restoration technology designed in this paper is suitable for murals in various damaged areas, can effectively restore the color of the image, can improve the accuracy of mural image restoration, can protect the integrity of mural image edge information, can optimize the effect of mural image color restoration, and is of some help to the development of mural culture in our country.
Data Availability
The authors can provide all the original data involved in the research.
Conflicts of Interest
The authors indicate that there was no conflict of interest in the study.