Abstract
A new river flow measurement method based on graphic process has been proposed recently, which gets the velocity in optical imaging modality through measuring the continuous displacement of floating debris, then reconstructs a two-dimensional river surface velocity field by using the velocity of floating debris, and computes the section flow at last. However, the surface optical images have not only lights of target information, but also surface optical noise. It is difficult for reliable and stable continuous displacement detection of complex small observation target, which occupies only a small number of pixels comparing to a large field imaging area and has complex optical reflection properties. To solve this problem, this paper presents a background suppression method based on soft morphology and Retinex theory. Soft morphology is firstly used for the opening operation of the image, and then Retinex theory is used for optimal estimation of image incident component to suppress background of image. Finally, the simulations show that our method is superior to gray morphology and soft morphology on the performance of targets enhancement, noise filtering, and background suppression, and it has better background and targets discrimination quality subjective evaluation and higher signal-to-clutter ratio.
1. Introduction
Image background suppression is the method of removing the relatively motionless background information from video frames and retaining the moving target information of a scene from video frames. Background suppression is widely used in image processing applications and is usually the first stage of the detection applications.
In the late 1990s, Fujita used particle image velocimetry (LSPIV) to observe floods in the Yodo River, which has always been used to put artificial tracer particles to the sink in laboratory conditions. The river flow measure method needs to capture images on the river bank of a tilt angle of the surface and measures large areas of natural rivers, gets the velocity in optical imaging modality through measuring the continuous displacement of floating debris such as tree branches, leaves, and other water tracers, then reconstructs a two-dimensional river surface velocity field by using the velocity of floating debris, and computes the section flow by velocity-area method at last. This measurement method was called large-scale particle image velocimetry [1–8] and has been applied widely in the field of hydrology.
The existing LSPIV method only follows the particle image enhancing technology of traditional PIV [9, 10], so that the signal-to-noise ratio (SNR) improvement in image is limited. However, the river surface imaging environment is very complex. There are many lighter components in the surface shooting image. The image is a collection of direct light from sun, atmospheric scattering light, surface reflected light (flare), surface-emitting light (reflection), underwater reflected light, targets reflected light, and so forth, so the background of the image is undulate. On the other hand, the flowing tracers that meet the following requirement on the surface have a smaller size. The targets are weak and there are small distinctions between background and targets, so the targets are difficult to be found. Therefore, the research on the methods of target enhancement and background suppression in the river surface image is important on the theoretical significance and application value [11–15].
This paper will propose one new background suppression method of the river surface image based on soft morphology and optimal estimation of image incidence component in Retinex theory. The related works and main contributions of the current work are presented in Section 2. The background suppression method of surface image based on soft morphology has been discussed in Section 3. Section 4 is divided into two subsections to explain processes of the method that combined soft morphology and Retinex theory and then propose our method. Section 5 gives the experiment results to verify the efficiency of our method by comparing the different methods. Finally, the paper is concluded in Section 6.
2. Related Works
Morphology was applied in background suppression several years ago, and the background suppression method based on mathematical morphology has been proposed a long time ago. The mathematical morphology method is developed from geometry and proposed by Serra and Soille [16], which is a nonlinear filter essentially. The method has unique advantages to remove the high-frequency components in the image. However, the traditional background suppression method of gray morphology is limited by the size selection of structuring element, and it is difficult to achieve the optimal suppression effect against the different target size.
Soft morphology [17] is developed from the gray morphology and proposed by Kuosmanen and Astola at first. The operating methods of gray morphology, such as erosion, dilation, opening, and closing, are based on the process with a single structuring element to achieve smooth background. However, it is not good to use one single structuring element in the complex background. Soft morphology is to reform the structuring element and let the single structuring element become a structuring element system. It uses the structuring element system that considers the difference between targets and target neighborhood to improve the effect of background suppression. And its performance is significantly improved compared to the standard gray morphology filter.
This paper will propose a new image background suppression method based on soft morphology. The complex river surface image was processed with soft morphology operation, and then the optimal estimation of image incident component to the result image was made. The results are also compared with two kinds of existing morphological background suppression methods.
3. Background Suppression Based on Soft Morphology
Soft morphology is developed from basic gray morphology. Gray morphology filtering method is usually used in the infrared image process. Now, we analyze the effect of the gray morphology filtering method in the river image background suppression.
3.1. Gray Morphology and Top-Hat Method
Gray morphology is the basis of traditional mathematical morphology. The most typical background suppression method of grayscale image is the top-hat transform [18–20]. The white top-hat transform and black top-hat transform (WTH and BTH) of grayscale morphology are as follows:
The first step of the process of and transforms is to make opening operation and closing operation to the original image and then make a difference between the operated results. The computational process of operation is always conducted below the original image , which is equivalent to the result that uses the original image to subtract the opening operation result. The computational process of operation is always conducted above the original image , which is equivalent to the result that uses operation result to subtract the original image .
Figure 1 shows the experimental results of and transforms. They are operated with 3 × 3 pixels’ square structuring element in surface grayscale images (256 × 256 pixels). From Figures 1(a2) and 1(a3), we can find that transform is able to detect the peak of images and transform is able to detect the valley of images effectively. In the background image of displacement detection, the targets look like small bright spots and will be lost in the image after transform. Compared with the opening and closing operations of grayscale morphology, the result of image background suppression after transform is obvious. It not only removes clutter effectively, but also retains the target integrity. The operation of can be regarded as a process of background estimation. The physical meaning of is a process that uses the original image to subtract the result after opening operation and retains the targets. So, the transform is a process of background suppression essentially.
From the comparison of gray distribution images after and transform, we find that the gray intensity of image background tends to zero except for the targets and noise. Figure 1 shows the experimental results of and transform. The original images are operated with 3 × 3 pixels’ square structuring element. Figure 2 shows the experimental results of and transform. The original images are operated with 7 × 7 pixels’ square structuring element. The targets have greater visibility after transform with 7 × 7 pixels’ square structuring element. The targets that are smaller than the structuring element size have been filtered in Figure 1(a2). The comparison of gray distribution shows that the targets have higher gray intensity, and the contrast of targets and background in the image is higher. Therefore, the size and structure selection of structuring element are very important for background estimation process. If the target size is too small, it causes elimination; if the target size is too large, it causes excessive smooth. These results will disturb the displacement detection of targets and motion vector estimation in the subsequent process.
3.2. Soft Morphology
In soft morphology, the maximum and minimum operation of standard gray morphology are instead of sorting weighted statistical operation. The determination of weighted coefficients is associated with the structuring element. Unlike standard mathematical morphology, the structuring element in soft morphology consists of two parts: one is the center of the structure (also called hard-core), and the weight (number of iterations) of corresponding pixel is greater than 1 or equal to 1; the other is soft edge , and the weight of corresponding pixel is equal to 1. The schematic diagram is shown in Figure 3.
Similar to the standard mathematical morphology, soft morphology erosion and dilation operations are defined as follows:
Formula (2) is the erosion operation of soft morphology, and formula (3) is the dilation operation of soft morphology. is the hard-core of structuring element; is the definition domain of ; is the soft edge of structuring element; is the definition domain of ; the structuring element , and ; is a repeating set; is the number of iterations. When , , and , the soft morphology is degenerated to the standard gray morphology. The center pixel of is also the center pixel of ; the soft edge pixels of are also the soft edge pixels of ; the weights of and are determined by ; the value of reflects differences between target area and the neighborhood. We can change the soft morphology structuring element by adjusting the value of and make the differences between targets and background in image to achieve a better effect of background suppression.
Based on erosion and dilation operation, the definition of soft morphology opening and closing operation is as follows:
Formula (4) is the soft morphology opening operation, and formula (5) is the soft morphology closing operation. The process of opening operation can be interpreted as making dilation operation to with soft edge first and then making erosion operation to the result with and . The process order of closing operation is contrasted with opening operation. Hard-core and soft edge are shown in
Figure 4 shows the results after the soft morphology operation, in which (b) is the opening operation result and (c) is the result of transform based on soft morphology opening operation. By using soft morphology opening operation, it can smooth the background image, filter targets and noise smaller than the structuring element, and get an estimated image of the original image background. From the comparison between Figures 4(c) and 2(a2), the background suppression effect of soft morphology top-hat () transform is better than gray morphology ’s. After transform filtering in Figure 4(c), the gray gradation of target is decreased obviously. The small weak targets almost are suppressed. In order to improve the background suppression effect of soft morphology, we propose a new background suppression method based on soft morphology filtering and Retinex theory.
(a) Original image
(b) Result of soft morphology opening operation
(c) Result of soft morphology background suppression
4. Background Suppression Based on Soft Morphology and Retinex Theory
Based on the analysis in the previous section, it can be found that the soft morphology operation has a good performance in suppressing background of the surface image. In order to enhance the image contrast and improve SCR in the image, we make the background suppression of displacement measurement of targets and motion vector estimation in the surface image with the Retinex theory.
4.1. Retinex Theory
Retinex is derived from the combination of retina and cortex. It is a color theory which can describe the color constancy of the human visual system. In the study of the principles of the human visual perception system and psychophysical brightness, Land [21] found that when the visual system processes the visual information, some uncertain external factors such as light intensity and uneven light will be excluded, and some characteristics information which can reflect the essence of objects will be retained. Then, these characteristics’ information will be delivered to the cortex by neural network and form visual image. Retinex theory was proposed in 1977; it can powerfully explain the homeostatic mechanism: the human visual system can achieve the same color of one object under different light by self-regulation.
Figure 5 shows the schematic diagram of Retinex theory. According to Retinex theory, the image is constituted by two factors. One factor is the illumination intensity of the object, which corresponds to the low-frequency part of the image and presents the luminance image ; another factor is the reflective brightness of the object, which corresponds to the high-frequency part of the image and presents the reflection image . So the imaging process of the image can be expressed as
The illumination intensity determines the dynamic range of pixel in an image, and the reflection luminance reflects the nature of the object. The essence of Retinex theory is casting aside the nature of the illumination intensity and obtains the inherent essential characteristics of the object from the image . Taking the logarithm of formula (7) into account, the complex operations can be translated into simple addition and subtraction, and the formulas are as follows:where , , and .
Usually, we cannot achieve the reflection luminance of the object directly. However, we can estimate the illumination intensity firstly. Then, we use the image to subtract the illumination intensity . In this way, the reflection luminance which can reflect the essential characteristics of the object can be achieved. The formula can be expressed as
This is also equivalent to the concept of background suppression principle: the high-frequency part (including target and high-frequency noise) can be separated by comparing the original image with low-frequency part of the image. Therefore, how to estimate the light intensity is the key of the issue.
Ferwerda et al. [22] showed that incident component in an image can be estimated and Kimmel et al. [23] showed that the incident component estimation problem (illumination intensity ) can be formulated as a Quadratic Programming optimization problem, and furthermore they showed the optimization problem having a unique solution. We will apply the above conclusion in our algorithm.
The commonly used methods to estimate the incident component include look-up table and convolution methods. To deal with the background suppression issue of the river water visual image, it needs to face multiple different images. Apparently, building a single gray look-up table cannot meet the requirements. Therefore, we use the method of the convolution operation to estimate the optimal incident component. In this method, selecting the appropriate kernel function to do the convolution operation is the key of the problem. Gaussian kernel function can highlight the center position of weight value. Meanwhile, the influence of the surrounding points of the center position can be taken into account. And the estimated image has a good correlation with the original image. Based on the above reasons, 3 × 3 Gaussian kernel function is chosen to do the optimal estimation of the incident component. The values of 3 × 3 Gaussian kernel function are as follows:
The convolution operation to the image with Gaussian kernel function is equivalent to doing a low pass filter. A new image will be achieved after each convolution and the optimal estimated value of incident component can be achieved. According to the literature [24, 25], the average gray level of the image tends to stability after three convolution operations. So it is thought that the result after the third convolution is the most suitable result to be the optimal estimation of the incident component of the image.
4.2. Background Suppression Method
In the process of target motion vector estimation with the river visible background image, the obtained river image has a complex background which includes many lights such as direct illumination from the sun, atmospheric scattering light, surface reflected light (flare), surface-emitting light (reflection), and target reflected light. Therefore, the image will present the uneven light, undulating background, and unidentified target. For the complex situation, a method based on soft morphology and Retinex theory is proposed to realize image background suppression.
According to the previous analysis, we can find that the estimation of incident component can achieve an optimal estimation for low-frequency part of the image. It has important practical significance of the surface visible background image with complex lighting conditions. Through the soft morphology operations and optimal incident component estimation, we can achieve the optimal estimation of the background image. Then, by using the original image to subtract the estimated image, a background suppression image with a higher signal-to-noise ratio will be achieved. The flowchart of the proposed method is shown in Figure 6.
Step 1 (opening operation). It has been shown that the optimized size of operator structure is generally equal to the half of the maximal size of a small two-dimensional target [26]. Therefore, we chose the following central structuring element and flexible edge structure element [27–29]:The structural element is composed of and . It takes soft morphological opening operation on the original image. After the opening operation, the noise and the target which is smaller than the structuring element in the image are eliminated, and the background image becomes smoother.
Step 2 (estimating the optimal incident component). According to the aforementioned method of optimal estimation of image incident component, we make a cubic convolution with the result after Step 1 and Gaussian kernel function in formula (10). Through Gaussian convolution, we filter out the high-frequency part of the image and achieve the optimal background estimation of the image.
Step 3 (background suppression). Using the original image to subtract the optimal estimated image obtained after Step 2, we can achieve the background suppression result that filters low-frequency part of the original image and enhances contrast:where is target component, is noise component, is original image, and is the background image after the processing of soft morphology opening operation and the incident component optimal estimation. Formula (12) can be as the soft morphology white top-hat transform based on Retinex theory.
5. Experiments and Simulations
In order to verify the effectiveness of the proposed method, we make an experiment with the method which combines with soft morphological opening operation and optimal estimation of incident component with background suppression of the image based on Retinex theory. We make a comparison between and . The experiment is carried out with the PC which is equipped with Pentium T4300, memory 2.86 GB, and uses Matlab software platform. To evaluate the performance of this method, we use pixel-level evaluation. In the study of image background suppression, an indicator known as SCR (signal-to-clutter ratio) is used as follows:where is the target strength, and it can be replaced by the maximum grayscale value of the image under the visible light; is the average grayscale value of the image, and it reflects the background DC component; is the grayscale standard deviation of the image, and it reflects the degree of clutter. The SCR Gain can be defined aswhere and denote the SCR of the original image and the background suppression image. SCR can be used to describe the improvement of the original image with the background suppression method. In experiment, we chose three screenshots from the actual shooting river scene video as subjects (image size is 256 × 256 pixels), as shown in Figures 7(a1), 7(b1), and 7(c1). Figures 7(a2)–7(a4), 7(b2)–7(b4), and 7(c2)–7(c4) are the results of the experiments.
Figures 7 and 8 and Table 1, respectively, present the result charts, three-dimensional grayscale distribution diagrams, and SCR/SCR Gain data statistics of the three surface image background suppression methods. Grayscale morphological transform is computed with 5 × 5 square structuring element, while the soft morphology white top-hat (SWTH) transform and the method in this paper are using the structuring element which is comprised of and in formula (11). From Table 1, it can be found that the original image has a low SCR, the targets have poor visibility in the grayscale image, and the grayscale values of background are fluctuant. WTH and SWTH transform can suppress background, eliminate some of the background clutter, and improve the SCR. However, there is still some background clutter residue in the image. Through the proposed method in this paper, the background clutter in the image is filtered to a large extent. The background is flattened, and the grayscale value of background is low. Moreover, most of the targets are retained and have good visibility, and the target enhancement is notable; these can be reflected by SCR Gain in Table 1.
Compared with the three methods mentioned in this paper, the proposed background suppression method has a notable improvement of SCR in the image, and the SCR Gain is also the largest. Grayscale morphology WTH transform has a weak performance in background suppression, and the soft morphology SWTH has a medium performance in background suppression. Therefore, the experiment fully demonstrated that the proposed method has a better ability in background suppression than the other two methods. However, this method also has some shortcomings. After the image processing of image 3, the effect of background suppression has no notable improvement compared with WTH and SWTH, and the three methods also have almost the same performance in SCR Gain. The reason is that the pixel size of the target in image 3 is large. In this method, the large size target with soft morphological opening operation cannot achieve an ideal result in background clutter residual. Therefore, the background image has a big fluctuation after background suppression. Although the improvement of SCR is not notable, the target grayscale and image contrast have a notable improvement. Moreover, our method has the largest grayscale value and optimal target visibility among the three methods, and the subjective evaluation of the quality in discrimination between the background and target is also the best.
6. Conclusion
To overcome the shortcomings of surface noise and clutter, surface tracer optical reflection complexity, difficulty in target displacement detection, and motion vector estimation, we present a background suppression method based on soft morphological filtering and Retinex theory in this paper. In order to improve the performance of surface image background suppression method, we use the Retinex theory and make an optimal estimation of incident component of the background image through soft morphological opening operation. The experiments give the results of background suppression of surface image and make a comparison with grayscale morphological WTH transform and soft morphology SWTH transform experiments. The simulations show that the proposed method has a notable improvement in background suppression of surface image. Meanwhile, our method makes a good preparatory work for the next target displacement detection and motion vector estimation.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This paper is partially supported by the National Natural Science Foundation of China (no. 61263029, no. 61374019), a project funded by the Priority Academic Program Development (PAPD) of Jiangsu Higher Education Institutions, and Natural Science Foundation of Jiangsu Province (no. BK20130851).