Abstract
Computer vision is currently playing an increasingly important role in automatically identifying the character of the image processing technology as research hotbed in the field of smart computing, OCR, face recognition, fingerprinting, biometric recognition, and so forth. Content-based image recovery, video recovery, multimedia collection, watermarking, games, film stunts, virtual reality, e-commerce, and other apps are available all round. The color pictures of parts taken by industrial cameras depend on computer performance and the intricate environment, and in particular, on the whole resolution image display, a lot of CPU resources are needed. Some details cannot be shown completely at the same time. If the image is not sufficiently clearly visible, methods for image processing like improvement, noise reduction, and interpolation must be used to improve color photo clarity. This article, based on the OpenCV platform, uses frequency domain filters, median filters, Fourier transform, and other image improvement technologies to remove image noise in order to enhance the quality of local photos from industrial cameras’ components. Finally, clear and available image information is obtained in different experimental methods, which check the application of image enhancement technology to image rebuilding. Finally, the performance of the proposed method in terms of CPBD value, definition value, and operation time is compared, which shows that the proposed method has obvious advantages in the above performance.
1. Introduction
In recent years, computer vision has been implemented in image recognition, cut, important information capture, and other areas as a consequence of the rapid development of deep learning and artificial intelligence technology. In 2014, the world-famous internet company recognized a cat from many photos through powerful computer resources which made great breakthroughs in computer technology and in the field of machine learning. With the promotion of algorithm progress, mainstream AI companies have reduced the data volume of identifying target objects from 10 million pictures to 80000, and data annotation and image processing methods play an indispensable role in the application of computer vision. In real life, it is necessary to take a lot of photos to collect a large amount of data. For industrial level data, it is difficult to ensure that all color pictures meet the standard. Therefore, it is necessary to process the pictures twice. In digital image processing, image restoration and reconstruction is an important work. Usually, the filter algorithm is used to remove the noise in the image, and the image enhancement technology is introduced to make the image have a higher resolution and clearer image. The shooting and recognition of industrial cameras in parts are typical applications. For some defective and fuzzy images, it needs to be processed separately. If the image is not clear enough, image processing techniques such as enhancement, noise reduction, and interpolation can be used to enhance the image display resolution.
Image enhancement refers to a series of methods to improve the display effect of images, or to transform the image into a form that is easier to process by machine. From the enhanced scope, it can be divided into two types: spatial domain enhancement and frequency domain enhancement. From the perspective of method, it can be divided into four categories: point operations (it contains image negative, contrast stretching, compression of dynamic range, gray level slicing, image subtransaction, and image averaging and histogram), mask operations (smoothing, medium filtering, sharping, and dynamic operations), transform operations (low pass filtering, high pass filtering, and high pass filtering), band pass filtering, homographic filtering, and sorting operations (false coloring and full color processing). Image enhancement can highlight the edge and important texture features, remove some noise, and suppress the display of unimportant areas, which improves the visual effect of the image to a certain extent. In fact, there are many factors that affect the quality of images, such as image degradation refers to the image quality deterioration due to the imperfection of imaging system, transmission medium, and equipment in the formation, storage, and transmission process. In order to make the enhanced image closer to the original image, it is usually necessary to analyze the reasons for image degradation, which may be caused by lens distortion, or because of motion blur, and also due to noise. Others such as the divergence caused by the optical characteristics of the imaging system, the image blur caused by noise and relative motion, and noise from circuit and photometric factors, images in space satellite, remote sensing, and astronomy will degrade due to atmospheric turbulence and relative motion between camera and object.
In the work of Kwon et al. [1], a Gaussian filtering method is proposed to enhance the image in time domain, which can improve the quality of image reconstruction. In image enhancement, edge detection and resolution enhancement are important problems in computer vision. Based on the principle of photon time stretching technology, literature [2] deduces the relationship between the contrast of the damaged image and the inherent nonlinear transfer function and proposes a visual transformation method to extract image features and enhance the display effect of the image. According to the causes of image damage, some researchers use different methods to classify the image according to the degree of blur, noise, and contrast. The typical factors are the image visual damage caused by the illumination conditions, the imaging situation of industrial cameras, and the changes of the surrounding environment. In view of the influence of fog in vehicle camera image detection, literature [3] proposed an image descriptor to distinguish whether there is fog in the photo, which solved the problem of fog image recognition and detection to a certain extent. In addition, some wavelet transform methods have also been applied to digital image processing. Literature [4] uses discrete wavelet transform and matrix singular value decomposition to reconstruct the image, which enhances the contrast of satellite image and achieves better results compared with traditional methods.
The second part of this paper summarizes the research status of image processing, the third part proposes Fourier transform and frequency domain filtering processing methods, the fourth part proposes to enhance the blurring of industrial images, and the fifth part carries out experimental verification and performance evaluation on the proposed methods.
2. Related Work
Recently, many machine learning technologies have been used in image processing field [5–7]. As image processing is the restoration and reconstruction of the original image, it has great application value in real life. Image enhancement can highlight the areas of interest, make the image outline clearer, and express more details. Therefore, there are many image processing [8–13] and enhancement signal processing technologies [14–16]. The traditional image enhancement technologies [17] are mainly in the form of manual extraction and construction of parameters, filtering, or transformation according to the noise and distortion degree of the image, and typical image processing technologies include GHE technology [18, 19], gray transformation, histogram, low-pass filtering, high pass filtering, and differential operation. Due to the influence of the weather environment, the rain image has low definition, accompanied by more noise and jitter delay. How to make the rain image clearer is a significant work. Literature [20] processes the image many times and obtains the single enhanced rain image through periodic filtering. For the captured image, in addition to the use of high pass filtering, channel calculation is also carried out for the processed image in each stage, so as to eliminate the rain trace and get a clearer map image. In addition, it is also necessary to enhance the display of color image. Literature [21], multiscale retinex (MSR), and guided filter (GF) are proposed for color image enhancement, and the contrast of the image is expanded by contrast lifting, so as to improve the calculation efficiency and display effect of true color image. With the rise of deep learning algorithm, some researchers try to use deep learning algorithm [22–26] to enhance the image. Literature [27] proposes a fast shot divide and glow (fsdg) network to segment and enhance the image. The effect of the image generated under weak light conditions is better than that of the artificial construction parameters. Literature [28] proposes a denoising algorithm for resisting high-frequency spatial interference to detect images with low definition. An end-to-end convolutional neural network is used to train the target image, and the target with enhanced image details is added to the loss function. The effectiveness of the method is verified on the data set, and this kind of network can detect the edge details in the image better and enhance the clarity of the image. Literature [29] proposes a convolutional neural network-based image illumination enhancement method, which uses light enhancement net to train the road scene image under low illumination conditions, converts the daytime image into weak light image, and reduces the risk of vehicle collision peak. In order to solve the problem of training data in deep learning image enhancement, an unsupervised network is proposed in literature [30], which introduces attention mechanism [31, 32] to focus on image loss sensing fusion and can enhance the image in low light level, at the same time, it can adapt to the real image enhancement display in different fields.
There are many application fields of image enhancement, such as medical treatment, industry, science and technology, and daily photo processing. Literature [33] aims at the problem that the image captured by computer in rainy day is fuzzy and proposes a rain image enhancement method based on joint depth neural network. This method mainly uses the encoder to recognize the rain stripes in the image and uses the multilayer convolution network and the relu activation function to enhance the image. To a certain extent, it solves the low-quality problem of taking photos in rainy days at night. Literature [34] is aimed at enhancing the clarity of medical images, highlighting the local and overall characteristics of medical imaging, making the model pay attention to the region of medical interest, and using the pretraining model RESNET and efficient net model training network to enhance the image. The problem of the low contrast of chest X-ray image of pulmonary tuberculosis was solved. A more interesting work is that literature [35] studies the influence of eyelashes, glasses frame, skin, and background noise in iris recognition. In order to better detect the iris boundary, they combined convolution neural network and capsule network to propose three filtering methods for fuzzy image enhancement, It improves the accuracy of the algorithm in iris recognition task. In the above research work, the image blurring is not considered in the image processing process, and the processing process is processed according to the general image processing flow. The enhancement image processing process proposed in this paper takes into account the general image enhancement process and image blur and can enhance the clarity of the image by denoising and interpolation.
3. Proposed Method
The image is not clear enough, so it should again be processed, due to the limited processing capacity and distortion in the lens of the industrial camera. Filtering the frequency domain can remove the noise from a frequency angle. This is a common method of image transformation and can improve the image display resolution. This paper contains information of the function in the frequency domain with the assistance of the two-dimensional Fourier transformation, and filters enhance the image.
3.1. Fourier Transform
Fourier, a great mathematician, proved that the aperiodic function can be expressed by sine/or cosine multiplied by the integral of the weighted function. Any time sequence or signal of continuous measurement can be expressed as the infinite superposition of sine wave signals of different frequencies. The Fourier transform algorithm based on this principle calculates the frequency, amplitude, and phase of different sine wave signals in the signal by using the original signal measured directly. It is also similar to the image, which can transform the time domain of signal processing to the space domain, map the one-dimensional information to the two-dimensional information, and transform the sine wave to the spatial frequency wave. Therefore, the Fourier principle can be applied to the two-dimensional image processing task.
The frequency of the image is an indicator of the intensity of gray changes in the image, and it is the gradient of gray in the plane space. The spectrum map obtained by the two-dimensional Fourier transform of the image is the distribution map of the image gradient. Of course, there is no corresponding relationship between the points on the spectrum map and the points on the image, even without frequency shift. In fact, the intensity of the difference between a certain point in the image and the neighboring point is the size of the gradient, that is, the size of the frequency of the point. For example, in the region of slow gray change in the image, the corresponding frequency value is very low; however, the frequency value is higher in the region where the gray level changes sharply.
For the frequency transformation degree of the image, a large gradient means that the brightness of the point is strong; otherwise, the brightness is weak. In this way, we can observe the spectrum after the Fourier transform, especially the energy distribution of the image. If there are more dark points in the spectrum, then, the actual image is softer (because the difference between each point and the neighborhood is not big, and the gradient is relatively small). On the contrary, if there are more bright points in the spectrum, then, the actual image must be sharp, and the boundary is clear and the pixels on both sides of the boundary are quite different. After shifting the frequency spectrum to the origin, we can see that the frequency distribution of the image is symmetrical with the origin as the center. In addition to clearly showing the frequency distribution of the image, shifting the frequency spectrum to the center of the circle has another advantage. It can separate interference signals with periodic regularity, such as sinusoidal interference. On a frequency spectrum with sinusoidal interference and shifting the frequency to the origin, it can be seen that in addition to the center, there is a set of symmetrically distributed bright spots with a certain point as the center, and this set is generated by the interference noise. At this time, the interference can be eliminated intuitively by placing a band stop filter at this position.
The form of two-dimensional discrete Fourier transform is as follows:
The corresponding inverse Fourier transform is as follows:
In the above expression,
The transformed amplitude spectrum is expressed as:
Two-dimensional Fourier transform has the property of linear addition. Fourier transform is a linear system. The Fourier transform of function sum is equal to the sum of Fourier transform of each function.
According to the similarity theorem, signal compression in time domain (, faster change speed) is equivalent to signal expansion in frequency domain (widening frequency band). On the contrary, signal stretching in time domain (, slower change speed) represents signal compression in frequency domain (narrowing frequency band). The symbol is as follows:
In the Fourier transform of image, it can be divided into amplitude spectrum and phase spectrum. Amplitude spectrum represents the amount of a certain frequency component in image, and phase spectrum represents the position of frequency component in image. In order to filter in frequency domain, we usually only care about amplitude spectrum, because from amplitude spectrum, we can see that bright line reflects the gray level change of original image, this is the contour edge of the image. The Fourier transform of the image is the transformation between the spatial domain and the frequency domain.
Discrete Fourier transform is an important tool of digital signal processing, which can transform the signal information in the time domain, but it also has some disadvantages, such as large amount of calculation and long operation time. As a result, the algorithm limits its application range to some extent.
3.2. Frequency Domain Filtering
With Fourier transform as the basis, the image can be enhanced by frequency domain filtering. Frequency domain image enhancement is to transform the image from spatial domain to frequency domain by Fourier transform and processes the frequency components of the image accordingly, so as to realize the function of image enhancement. The spatial image enhancement based on Fourier transform is a linear system filtering:
By convolution operation of Fourier transform, the above linear system can be transformed into
Among them, represents the Fourier transform of the image to be enhanced; the Fourier transform , which represents the unit impulse response of the spatial linear system, is called transfer function or filter function. Fourier transform is used to change the amplitude spectrum (without changing the phase spectrum) of the frequency domain image of so as to realize the frequency domain filtering enhancement of the image captured by the industrial camera
The principle and steps of image enhancement by frequency domain filtering are as follows: (1)Preprocess the image (0 fill)(2)Move the special pixel to the center(3)Perform Fourier transform operation on the image:(4)Filtering of the original image(5)Perform the inverse Fourier transform operation again:(6)After filtering, the special pixels are shifted back:(7)After filtering, extract the left quadrant
Through the above 7 steps, the enhanced image after frequency domain filtering can be extracted with the help of Fourier transform. Frequency domain filtering can obtain global filtering enhancement information, obtain more ideal results and deal with more complex clutter removal multifeature enhancement, construct a filter, deliberately boost some frequency components, and depress or remove other components, so as to achieve the purpose of image enhancement, as shown in Figure 1.
3.3. Image Enhancement Processing of Color Photos of Components Taken by Industrial Cameras
When the local color image of components captured by industrial camera is not clear enough, the image needs to be denoised and interpolated to enhance the resolution of the image. Because frequency domain filtering can directly process the frequency spectrum and reconstruct the display effect of the image, this paper proposes a frequency domain filtering based on Fourier transform to enhance the true color image information of components. The goal is to build a low-pass filter, which can effectively prevent the high-frequency components and allow the low-frequency components to pass smoothly, so as to filter out the noise in the high-frequency part of the frequency domain, and then the enhanced image after smooth denoising can be obtained by inverse transform.
Low pass filtering needs to calculate and intercept the frequency of the graph. First, it needs to calculate the total signal energy PE,
The filtering power component of the calculated image is as follows:
In with low definition image whose coordinates are , the image filtered in frequency domain is as follows:
4. Experiment and Results
4.1. Experimental Realization
Based on the principle of frequency domain filtering for image enhancement, we randomly selected 100 true color images of components from the blurred images captured by industrial cameras and carried out experiments on the OpenCV platform to obtain the images after low-pass filtering. In Figure 1, the image on the left is taken by an industrial camera due to the influence of noise. From the image, the global information becomes very fuzzy and cannot reflect the real information of the image. Therefore, image enhancement is needed. In the experiment, the texture information of the image can be restored and reconstructed by using Fourier transform and frequency domain filtering. The reconstructed image is shown in the right figure, after filtering and enhancement, the global image information can be clearer, and the edge information can also be highlighted, as shown in Figure 2.
(a)
(b)
4.2. Performance Evaluation
At present, the evaluation methods for blurred image enhancement technology, because of the essence of images, restore some detail information or edge information for blurred images or low-resolution images. Therefore, this paper chooses Cumulative Fuzzy Detection Probability Measurement Index (CPBD) and Clarity Q Index for comprehensive evaluation.
CPBD index is a contrast measurement index based on cumulative blur detection probability without any reference to other standard images, which is the real response to blurred images after processing under visual perception. The experimental results show that there is a good correlation between this index and human consciousness evaluation.
JNB is defined as the minimum amount of perceived blur around the edge when the contrast is higher than the perceptible difference. For a given contrast, the probability of edge blur detection is defined as:
where represents the edge pixels of the blurred image; the parameter is empirically set to 3.45. is the measured width of the edge pixels of the blurred image; is the directly perceivable blurred edge width of the edge pixel of the blurred image, and its size is related to the local contrast around the edge. The calculation formula is defined.
where is the local contrast of the blurred image, which is the difference between the maximum and minimum values in the edge area of the image.
Formal model description of CPBD:
Another indicator is clarity, which indicates the recovery of blurred images on detail pixels, generally showing the size of resolution and edge arts and sciences. This information can explain the image nodule recovery ability under enhancement technology from another side, described as:
where represents the definition measure value of the image, and and represent the pixels of the image, respectively. It is described by the following formula:
where is the original image pixel value at the image channel ; represents the maximum pixel value of the original image.
4.3. Experimental Performance Comparison
In order to reflect the implementation effect of this algorithm, a variety of blurred images are used as experimental objects, and the performance under different images is compared under three performance indexes: CPBD value, definition value, and operation time. Figures 3 and 4 show the performance comparison under different blurred images.
Figures 3 and 4 show that the CPBD value of this algorithm is obviously higher than the other two algorithms, which shows that the performance of this algorithm is the best in contrast improvement; the clarity value of this algorithm is also the largest, which shows that the performance of this algorithm is the best in detail information recovery. In terms of the operation time, the algorithm in this paper takes the shortest time, which is much faster than the DCP algorithm.
Compared with the above performance, the performance advantage of this paper is very obvious. Therefore, this paper uses multiple groups of blurred images to analyze the performance and then takes the average value. The effect is shown in Figures 5–7.
Figures 5–7 show that the algorithm proposed in this paper is compared with CLAHE and DCP algorithms. The algorithm in this paper is obviously higher than the other two algorithms in three performance indexes, which shows that the algorithm has the best performance in contrast improvement; CLAHE is equivalent to DCP in CPBD performance. CLAHE has the worst performance under index, while DCP has the worst time performance under Time Index.
5. Conclusion and Prospect
Aiming at the problem of unclear parts pictures taken by industrial cameras, this paper proposes Fourier transform and frequency domain filtering as algorithms to enhance the image to perform secondary processing on the photos, so that the filtered image is highlighted. The quality of the image is improved. Compared with the original image, the effect of visual effects on the quality of the final image. This method can show the texture information of the target image more clearly. Due to the limitation of computing resources, we did not use deep learning to automatically process image information. In the future, the images we need to process are more complex, which may be image reconstruction in stereo scenes or image shadow noise removal under fast motion. Try to use convolutional neural network and semantic segmentation algorithms to further process the image, let the algorithm automatically acquire and learn the reconstruction method of the image, and enhance the contrast of the image, which can be widely promoted in practical applications.
Data Availability
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Conflicts of Interest
The authors declared that they have no conflicts of interest regarding this work.