Abstract
This paper presents the implementation of an algorithm for automatic identification of drops with different sizes in monochromatic digitized frames of a liquid-liquid chemical process. These image frames were obtained at our Laboratory, using a nonintrusive process, with a digital video camera, a microscope, and an illumination setup from a dispersion of toluene in water within a transparent mixing vessel. In this implementation, we propose a two-phase approach, using a Hough transform that automatically identifies drops in images of the chemical process. This work is a promising starting point for the possibility of performing an automatic drop classification with good results. Our algorithm for the analysis and interpretation of digitized images will be used for the calculation of particle size and shape distributions for modelling liquid-liquid systems.
1. Introduction
Image processing is a very relevant area of computer science with applications in many domains. Quantitative analysis and interpretation of digitized images are currently important tools in several scientific domains.
The acquisition and treatment of images of particulate phases become essential for the calculation of particle size and shape distributions, namely in multiphase systems modelling in chemical engineering. In particular, modelling and validation of liquid-liquid systems, either for hydrodynamic or mass transfer phenomena, can benefit from image processing techniques. This is of importance in simulation, interpretation, and performance predictions of multiphase reactors. As reported in Ribeiro et al. [1], authors like Olney [2], Cruz-Pinto and Korchinsky [3] and Rod and Misek [4] have demonstrated that serious design and performance prediction errors occur if drop size distribution is neglected. According to Pacek et al. [5], any technique based on representative physical sampling will drastically change the overall composition of the dispersion.
In Ribeiro et al. [1] and Ribeiro [6] a video technique with nonintrusive probes was tested. Pictures of a small region inside a transparent vessel near its wall were obtained, by lighting and observing it from the outside. In that work, to obtain the drop size distribution, images were analyzed by employing visual/manual techniques which imply high costs, intensive labour, weariness buildup, and consequent high error rates. A fully automated computational approach has a definite potential for better performance.
The aim of the present work is to develop an algorithmic process capable of performing shape discrimination and size classification for liquid drops in monochromatic digitized frames of a liquid-liquid dispersion.
In order to automatically identify the contour of the drops, some known techniques for edge detection in images have been tested. Preliminary results with the Sobel, Marr-Hildreth, and Canny [7] methods were not satisfactory.
Therefore, we have developed a new approach which is described in this paper. After preprocessing the images, we have used Hough transforms [8] for the detection of round drops.
2. Experimental Setup and Noninvasive Image Acquisition Technique
Image frames were obtained at our SIPROM (Modelling and Simulation of Multiphase Systems) Laboratory with a digital video camera + microscope + illumination setup from a dispersion of toluene in water within a transparent mixing vessel [1] as shown in Figure 1.
The mixer consists of a 6,28 L glass vessel (diameter = height) with flat bottom, equipped with four flat vertical baffles. The agitation was provided by a standard turbine with a 1/2 turbine/vessel diameter ratio and with turbine diameter/disk diameter/paddle width/paddle height of 20/15/5/4. The mixer-settler arrangement works in closed circuit with the mixer feeds being forced in by peristaltic pumps.
For our experiments, the average residence time of the mixture in the mixer vessel was between 1 and 10 minutes, the dispersed phase hold-ups between 1 and 10%, and the agitation speed between 90 and 145 rpm.
The images were captured by a black and white SensiCam [9] camera, designed for weak lighting and fast movement (exposure times from 1 millisecond to 1000 seconds, image intervals between 0 and 1000 seconds).
Due to the high sensitivity of the camera, the lighting system had to respond to strict requirements of target ability, no flicker, high light density, low heat generation and simple and safe setup. So the light system was made up of four cool halogen lamps (150 W, 12 V each) with mirror-concentrated beam and rear cooling.
Test trials enabled the definition of the best placement of the microscope-camera group, the best placement of the lighting array and the penetration depth (up to 3 cm) into the vessel. The amplification of the combined optical image system was determined from images of a gauge made from calibrated wire immersed in the vessel. In this routine work was used a minimum frame duration (1 millisecond) in order to disable drop trails even at the highest agitation speeds. In this way, many frames (up to 1000) may be obtained per second, which enables the selection, in each frame, of only the best-defined drops without representative sample size problems.
3. Definition of the Problem
The experimental conditions (mainly phase ratio and agitation speed) led to frames of deeply focused fields with partially overlapping drops and high background noise, as we can see in Figure 2. In this figure we show two images, Figure 2(a) is what we consider a better quality image whereas Figure 2(b) is very difficult to process. Since noninvasive image acquisition and lighting were our uncompromising starting option, no significant improvements were obtained although all available image acquisition and cleaning techniques were used within this fundamental constraint. Thus, partial images and ill-defined drop boundaries led to all low cost, market-available image processing software packages requiring a high number of frames in order to secure statistically significant drop samples which, again, made for greatly time-consuming, unreliable procedures.
(a)
(b)
This led us to develop our own software for which a preliminary test and calibration stage was performed on archive images previously obtained with a semiautomated procedure [1]. We have developed a promising approach, implemented in MATLAB [10], which, given one of these photographic images of a dispersion, automatically identifies the contour of existing drops and classifies them according to their diameter.
4. Description of the Method
Given one of the photographic images of the dispersion, our approach, implemented in MATLAB, automatically identifies the contour of existing drops and classifies them according to their diameter.
In our proposed approach, the process for the detection of the drops in an image has two distinct steps. In the first step, we detect the edges of the drops in the original image by monitoring the values of the gradient and the descending thickness and by creating an output image with those contours. In the second phase, we detect the drops in this contour image, using the Hough Transform [8, 11]. This transform is widely used in image processing to detect lines and also to detect circles.
4.1. Edges Detection
In Figure 3, we show the step of the detection process of the drops. Thus, starting from the original image (Figure 3(a)), we detect the edges (Figure 3(b)) and in the next step, we detect the contours of the drops using the Hough transform (Figure 3(c)).
(a)
(b)
(c)
In our images, of relatively poor quality, the drops have, in majority, dark edges.
To reduce the noise, and consequently reduce the probability of false edges detection, the original image (Figure 5(a)) is smoothed using a Gaussian filter. This eliminates some false contours and reduces the detection of false drops.
Using the typical profile of the signal, edge detection is made by evaluating the relation between the gradient’s descent (Gd) and its respective thickness (Ed), shown in Figure 4.
(a)
(b)
(a)
(b)
(c)
The filtered image (If) is derived, with kernels of convolution, as indicated below, originating two matrices, Ix and Iy (see (1)) corresponding to the partial derivatives and : From the images Ix and Iy we obtain the average and the standard deviation of the descending gradient and the average and the standard deviation of the descending thickness . The edge detection threshold is defined by the combination of Gd and Ed (computed from the Ix and Iy images), where Gd takes negative values and Ed positive.
In the processing of each descent (negative derivative) from its origin to inflexion point, if and , then the pixel of this point is considered an edge pixel. From the matrix Ix we obtain the partial contour matrix Ia, and from Iy the matrix Ib.
Since there may be very high values of and very low values of , the detection of the edge pixels could be negatively affected. To reduce that possible effect we have introduced the terms and . By moving the edge detection thresholds in this way, we intend to obtain the maximally useful information without increasing significantly the risk of obtaining unnecessary information.
The sum of Ia and Ib yields the contour image Ic, shown in Figure 4. Currently, we only consider the vertical and horizontal derivatives. Taking other directions into account (such as 45 degrees derivatives) could saturate the contour detection and degrade the results. Moreover, this would increase significantly the computational effort of the method.
Preliminary experiments with other known methods for contour detection such as Sobel, Canny, and Marr Hildreth showed that these degrade the results of the Hough Transform in the detection of the drops. They introduce additional information that causes the detection of false drops.
4.2. Detection of Drops in the Contour Image
In the second step of the work, we applied the Hough transform to the contour image Ic (Figure 5(b)) to detect the contour of the drops for different values of the radius.
We consider, as we can see in Figure 5(c), that the contour of each drop can be represented as one object with irregular form, centred in a point and with a radius that varies from to .
The Hough transform is widely used to detect lines and also to detect circles. To find circles using a Hough transform, each edge element votes for all the points which are centres of the circles with radius r that it could lie on. This transform allows determining the centre of the drop through the identification of the most voted zone. It has the inconvenience of, for each useful vote, generating noise votes. For this reason, in an image with many drops, this could cause the detection of inexistent drops.
As referred above, in this step of the process we applied the Hough transform to the contour image Ic taking into account the relative deformation of the drops. The value of k is a parameter of the algorithm. For a maximum deformation of radius the minimum deformation will be . Votes will be generated from the successive application of the Hough Transform to the range .
This procedure is repeated for all the radii considered as maximum deformation (), in order to process all the contours, of all dimensions, of the drops (see Figure 5(c)).
The voting window (centre of Figure 6) has a dimension of pixels for radii above 15 and dimension of otherwise. To reduce the number of noise points we considered half of the voting amplitude (two quadrants). For that we analyze the signal of the partial derivatives according to the partial contour matrix, Ia or Ib, related to the edge pixel.
Figure 7 illustrates the detection quadrant voting procedure for an edge pixel related to matrix Ia. If the drop was a regular solid, it would be possible to determine the correct quadrant, by analyzing the gradient’s angle variation, . In our case, it is only possible to determine a pair of possible voting quadrants (A, C or B, D), by analysing the partial derivative’s signal. The similar process is used for the edge pixels related to matrix Ib.
(a)
(b)
The detection process starts with the set of drops with radii between 17 and 48 pixels (0.17 and 0.48 millimeter, resp.) because these are the most frequent. Then, we treat the drops with at least 48 pixels of radius. Finally, we process the drops with radius between 8 and 16 pixels (0.08 and 0.16 millimeter, resp.).
Processing the drops with higher radius first reduces the probability of detecting false drops with a lower radius, caused by agglomerates of pixels in the contour of the drops. However, the detection is not efficient for radii below 8 pixels (0.08 millimeter).
This detection process is enhanced with the erosion of the contour images of detected drops. This is not a common process of morphologic erosion [11]. In this process, after finding a centre, we eliminate the pixels that have contributed to find this centre so that they do not interfere in the subsequent process [12, 13].
In Figure 8, we can see the results obtained on two images for a radius ranging from 5 pixels (0.05 millimeter) to 46 pixels (0.46 millimeter).
(a)
(b)
5. Results
To evaluate our approach more objectively we have chosen two images with different levels of photographic quality, and compared the sets of drops obtained automatically with the drops manually identified by us. Each image contains more than 100 drops with varying conditions (radii, overlapping, border quality, etc.). In Table 1, we show the results obtained with two of the images, the images shown in Figure 8, in terms of recall and precision values. We have calculated those values as defined in (2) where TP is the true positives (number of drops correctly identified by the program), FN the false negatives (actual drops not identified by the program) and FP the false positives (drops incorrectly identified by the program). In other words, recall measures the proportion of existing drops that the program was able to identify correctly, whereas precision is the proportion of drops identified by the program that are truly correct: In the case of the first image, we have obtained the following results. For a radius below 7 pixels the program cannot find any drop. For the radius 10 (0.1 millimeter), not in the table because and precision not defined, we have , , and . For the radius 12 (0.12 millimeter), also not in the table, because and precision are not defined, we have a , and . For the other values of the radius not represented in the table, we have , , and . As final results, taking into account all the values of radius, we have for this image a recall of 0.71 and a precision of 0.89.
For the second image, which has a lower photographic quality, we have worse results, having a total 0.55 for recall and 0.87 for precision. Nevertheless we have, for many values of radius in the image, maximum recall and precision. For several values of radius from 7 to 23 and for radius equal to 35, 40, and 45, recall and precision are not shown in the table, for the same reasons as in the previous image. We believe the cause for these worse results with respect to the previous image is the lower quality of this second image.
6. Conclusions and Future Work
In this paper, we have presented a method for the automatic identification of drops in images taken from agitated liquid-liquid dispersion. The results obtained with two images with more than 200 drops with diverse conditions (radii, overlapping, border quality, etc.) lead to the conclusion that our program is able to detect a good percentage of the drops. In the case of a better quality image, the program recognized 71% of the drops. For the other image, with lower quality, only 55% were detected. We have also observed that the approach is less efficient for smaller values of the radius, since very small drops can be easily mistaken by noise.
This work is a promising starting point for the possibility of performing an automatic drop classification with good results. However, given the limited number of images used in these experiments, these results must be further validated. Currently, we are manually identifying drops in our large library of images. Having a larger number of annotated images, we can proceed with experimental validation and further improve our results.
This can be done by fine tuning the parameters of the process, and learning the appropriate parameters given the image conditions (quality, lighting, etc.). Another line of research we are pursuing implies employing neural networks for strengthening drop recognition.